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Abstract 

This  reseaich  effort  was  conducted  to  study  the  effects 
that  operational  experience  acquired  through  the  Broadening 
Experience  Tour  (BEST)  program  has  on  the  ability  of 
scientific  and  development  engineering  (S&DE)  officers  to 
develop  useful  operator-oriented  systems.  A  review  of  the 
related  literature  revealed  that  although  the  Air  Force 
encourages  S&DE  officers  to  acquire  operational  experience, 
there  is  no  empirical  evidence  of  the  reported  benefits  of 
this  experience.  A  cross  sectional  study  was  undertaken  to 
measure  the  difference  in  self  reported  competency  between 
samples  of  S&DE  officers  with  and  without  operational 
experience.  A  survey  was  designed  to  measure  the  ability  of 
the  S&DE  officers  to  interpret  operational  requirements,  to  ' 
communicate  with  the  using  organizations,  and  to  measure  the 
credibility  of  their  personal  judgement.  The  results  of  the 
study  revealed  the  operational  experience  acquired  through, 
the  BEST  program  enhanced  the  ability  of  S&DE  officers  to 
communicate  with  the  using  organ  1  cat  ions  .  hated  officers, 
ranked  themselves  higher  than  BEST  oiricers  in  their  ability 
to  interpret  operational  requirements.  No  difference  was 
found  in  the  level  of  credibility  between  officers  with  and 
without  operational  experience. 
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AN  ANALYSIS  OF  THE  BENEFITS  OF  OPERATIONAL  EXPERIENCE 
ACQUIRED  BY  AIR  FORCE  SYSTEMS  COMMAND  (AFSC)  OFFICERS 
THROUGH  THE  BROADENING  EXPERIENCE  TOUR  (BEST)  PROGRAM 


I . _ Introduction 


General  Issue 

During  the  last  decade,  stories  of  cost  overruns,  spare 
parts  overpricing,  and  poor  quality  have  prompted  efforts  to 
improve  the  services  provided  by  the  Department  of  Defense 
(DoD)  acquisition  community  (1:21).  Consistent  with  this 
objective  the  Air  Force  created  the  Acquisition  Management 
Professional  Development  Program  ( AMPDP)  for  officers  in  the 
scientific  and  development  engineering  (S&DE)  career  fields. 
Air  Force  Systems  Command  Regulation  36-5,  which  outlines 
the  requirements  of  the  AMPDP,  states  that  "operational 
experience  gained  in  an  Air  Force  major  air  command  (other 
than  AFSC  or  AFLC )  or  in  a  joint  command  is  highly  desirable 
(2:2)."  Air  Force  Regulation  (AFR)  36-23,  Officer 
Professional  Development,  states  that  a  mixture  of  technical 
and  operational  experience  "will  enhance  the  ability  of  tile 
S&DE  force  to  interpret  operational  requirements,  develop 
useful  operator-oriented  systems,  communicate  effectively 
with  using  organizations,  and  (in  the  final  analysis) 
establish  credibility  for  their  judgment  (3:70)." 
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I.  Introduction 


Goner1.!  Issue 

During  the  lust  decade,  stories  or  cost  overruns,  spore 
parts  overpricing,  and  poor  quality  have  prompted  cl  torts  to 
improve  the  services  provided  by  the  Department  ot  Deier.se 
(DoD)  acquisition  community  (  I  :  C  1 )  -  Consistent  with  this 
objective  the  Air  Force  created  the  Acquisition  Management 
Professional  Development  Program  ( AMP DP)  for  officers  in  the 
scientific  and  development  engineering  (SSDE)  career  fields. 
Air  Force  Systems  Command  Regulation  36-5,  which  outlines 
the  requirements  ot  the  AM PDF ,  states  that  "operational 
experience  gained  in  an  Air  Force  major  air  command  (other 
than  AFSC  or  AFLC)  or  in  a  joint  command  is  highly  desirable 
(2:2)."  Air  Force  Regulation  (APR)  3o-23,  Officer 
Profession.!  j  Dove  1  onmep.t. ,  states  that  a  mixture  of  technical 
and  operational  experience  "will  cnh.mco  t ho  ability  oi  the 
S&DE  force  to  interpret  operational  requirements,  develop 
useful  operator-oriented  systems,  ecninun  i  cate  effectively 
with  using  organ  i  .’.at  i  ens ,  and  (in  the  final  analysis) 
establish  credibility  tor  their  judgment  ( j  :  V  0 )  ." 


This  research  project  examined  the  impact  of 
operational  experience  acquired  by  Air  Force  Systems  Command 
officers  through  the  Broadening  Experience  Tour  (BEST) 
program  on  their  ability  to  develop  useful  operator-oriented 
systems.  It  was  assumed  in  this  study  that  the  ability  to 
interpret  operational  requirements,  the  ability  to 
communicate  with  the  using  organization,  and  credible 
judgement  were  prerequisites  to  develop  useful  operator- 
oriented  systems.  The  S&DE  utilization  fields  associated 
with  R&D  are  Air  Force  Specialty  Code  (AFSC)  26XX  - 
Scientific  utilization  field,  AFSC  27XX  -  Acquisition 
Program  Management  utilization  field,  and  AFSC  28XX  - 
Development  Engineering  utilization  field. 

Background 

During  the  mid  70' s  most  of  the  Air  Force  officers  with 
operational  experience  were  assigned  to  acquisition  duties 
through  the  rated  supplement  (4:23)  .  The  rated  supplement 
is  a  system  that  "takes  pilots  and  navigators  out  of 
airplanes  (especially  it  there  are  far  too  many  rated 
officers  for  the  available  aircraft)  and  tries  to  utilize 
them  in  other  jobs  in  the  service  (5:21)."  These  officers 
were  rated  senior  captains,  majors,  and  lieutenant  coicr.els 
who  were  available  after  the  Air  Force  began  the  withdrawal 
from  Vietnam.  In  1 7u  officers  in  the  rated  supplement 
accounted  for  22  pat  cent  of  all  acquisition  cf f ieors  in  Air 
Force  Systems  Command  (1:2 1). 


During  the  late  70 's  the  number  of  officers  in  the  Air 
Force  decreased  due  to  the  "erosion  of  benefits"  and  the 
belief  "that  their  fellow  citizens  held  them  in  low  esteem 
(4:23)."  By  1987  the  demographic  composition  of  acquisition 
officers  had  changed  dramatically.  In  that  year  30  percent 
of  the  acquisition  officers  in  Air  Force  Systems  Command  had 
operational  experience,  out  of  which  only  six  percent  were 
officers  in  the  rated  supplements.  l'n  addition,  the 
acquisition  force  was  younger  as  well;  45  percent  of  the 
acquisition  officers  were  lieutenants  and  32  percent  were 
captains  (4:23) . 

In  1984  the  Air  Force  Broadening  Experience  Tour  (BEST) 
program  was  created  to  increase  the  number  of  officers  with 
non-rated  operational  experience,  and  to  fill  critically 
undermanned  operational  career  fields  (6).  The  BEST  program 
places  acquisition  officers  into  operational  assignments  and 
brings  them  back  to  Cisterns  Command  at  the  end  of  the  tour 
(4:26).  Currently,  operational  experience  is  infused 
through  the  rated  supplement;  crossflow  assignments  from 
operational  utilization  fields  into  S&DE  utilization  fields; 
and  the  BEST  program.  The  rated  supplement  currently 
provides  the  majority  of  officers  with  operational 
experience  ( 7 ) . 

Problem  Statement 

The  BEST'  program  was  created  to  increase  the  number  of 
officers  with  operational  experience  assigned  to  ssi'E 
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utilization  fields  in  Air  Force  Systems  Command.  A  mixture 
of  technical  and  operational  experience  is  believed  to 
enhance  the  ability  of  officers  in  S&DE  utilization  fields 
to  develop  useful  operator-oriented  systems.  Is  the  ability 
to  develop  useful  operator-oriented  systems  different 
between  officers  who  participated  in  the  BEST  Program  and 
officers  without  operational  experience?  If  officers  who 
participated  in  the  BEST  Program  are  more  able  in  general  to 
develop  useful  operator-oriented  systems  than  officers 
without  operational  experience,  to  what  extent  is  their 
advantage  due  to  the  relevance  of  their  operational 
experience  to  their  current  assignments? 

Cbi ective 

The  objective  of  the  research  was  to  determine  if  the 
operational  experience  acquired  through  the  BEST  program  by 
officers  in  the  26XX,  2.7XX,  and  28XX  career  fields  enhances 
their  ability  to  develop  useful  operator-oriented  systems. 

Investigative  Questions 

To  determine  the  impact  of  operational  experience 
acquired  by  AFSC  officers  through  the  BEST  program  on  their 
ability  to  develop  useful  operator-oriented  systems,  the 
following  questions  were  researched: 

1.  Dees  participation  in  the  BEST  Program  enhance  the 
ability  of  S&DE  officers  to  interpret  operational 
requirements? 


2.  Does  participation  in  the  BEST  Program  enhance  the 
ability  of  S&DE  officers  to  communicate  with  the  using 
organizations?  Using  organization  is  defined  as  the 
recipient  organization  of  the  system. 

3.  Does  participation  in  the  BEST  Program  help  S&DE 
officers  establish  credibility  for  their  judgment? 

4.  To  what  extent  is  the  ability  to  develop  useful 
operator-oriented  systems  affected  by  relevant  operational 
experience?  Relevant  operational  experience  is  defined  as 
experience  with  or  related  to  the  system  being  developed. 

Scope  and  Limitations 

This  research  focused  on  the  utilization  of  officers  in 
the  2 CXX ,  2 7 XX ,  and  28XX  career  fields  only.  Although  it  is 
possible  other  acquisition  related  utilization  fields  might 
benefit  from  the  infusion  of  operational  experience,  the 
time  allotted  for  the  completion  of  this  research  made  it 
impossible  to  include  them  all  in  the  study.  Since  the  BEST 
program  is  an  Air  Force  Systems  Command  program,  this 
research  was  further  limited  to  officers  in  this  command. 

In  addition,  this  research  was  confined  to  the  Aeronautical 
Systems  Division  (ASD)  which  has  the  highest  concentration 
of  BEST  officers  who  have  returned  from  operational 
assignments.  The  effects  of  operational  experience  on 
senior  S&DE  officers  were  not  addressed  in  this  study. 

Assuxpt ions 

The  key  assumption  in  this  study  is  that  the  ability  to 
develop  useful  operator-oriented  systems  is  dependent  or.  the 
ability  to  interpret  operational  requirements,  the  ability 
with  the  us  i  ng  organization,  anil  possession 
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of  credible  judgement.  Air  Force  Regulation  36-23  lists  the 
ability  to  develop  useful  operator-oriented  systems  at  the 
same  level  as  the  other  benefits. 


Jusrif icat ion 

Air  Force  Systems  Command,  like  the  other  Air  Force 
major  commands,  has  to  compete  for  people  with  the  right 
qualifications  at  a  time  when  budget  reductions  threaten  a 
reduction  in  military  force.  Since  officers  with 
operational  experience  in  acquisition  related  career  fields 
represent  a  minority,  the  effects  of  operational  experience 
should  be  studied  in  an  effort  to  determine  if  the  infusion 
of  the  operational  point  of  view  to  the  acquisition  process 
is  in  fact  beneficial  and  hence  should  be  pursued  as  a 
corporate  objective. 

Research  Approach 

A  literature  review  and  a  mail  survey  were  used  to 
answer  the  research  questions.  An  extensive  literature 
review  of  Air  Force  and  DoD  regulations,  and  program 
management  journals  was  conducted  to  obtain  the  secondary 
data  necessary  to  understand  the  requirements  and  purported 
benefits  of  operational  experience.  Interviews  with  HQ 
AFSC/ DPROA-2  personnel,  and  other  personnel  who  worked  in 
that  organization  during  the  start  of  the  BEST  Program  were 
conducted  as  required  to  provide  additional  information  not 
contained  in  the  published  material.  A  survey  was  then  used 
to  obtain  the  primary  data  necessary  to  determine  the 
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relevance  of  operational  experience  in  the  development  of 
useful  operator-oriented  systems. 

Summary 

This  chapter  presented  the  objective  of  the  study 
effort,  background  of  the  problem  area,  and  the 
investigative  questions  that  were  pursued.  The  limitations 
of  the  study  and  assumptions  were  also  discussed.  Chapter 
II  provides  the  result  of  the  literature  review  conducted  to 
gain  insight  into  the  problem  area.  The  details  of  the 
methodology  employed  to  answer  the  investigative  questions 
are  presented  in  chapter  III.  Chapter  IV  provides  the 
results  of  the  study.  Finally,  the  conclusions  and 
recommendations  are  provided  in  chapter  V. 


II . 


Literature  Review 


Introduction 

This  literature  review  describes  the  purported  benefits 
of  operational  experience  in  the  scientific  and  development 
en jineering  (S&DE)  utilization  fields  in  support  of  systems 
acquisition  in  the  Air  Force,  and  identifies  the  type  of 
assignments  that  qualify  as  operational  experience.  The 
main  sources  of  information  include  Air  Force  Regulations, 
government  program  management  journals,  academic  journals, 
and  telephone  interviews  as  required. 

Discussion  of  the  Literature 

Science  &  Development  Engineering  Utilization  Fields . 

Air  Force  specialty  codes  (AFSCs)  are  used  to  identify 
the  different  jobs  in  the  Air  Force  as  well  as  the  required 
qualifications  to  fill  these  jobs.  All  AFSCs  are  listed  in 
Air  Force  Regulation  .,6-1,  Officer  Air  Force  Specialty.  AFR 
36-1  also  specifies  knowledge,  education,  experience,  and 
training  requirements,  as  well  as  other  job  specific 
qualifications  for  each  AFSC  (8:3). 

An  examination  of  the  purported  benefits  of  operational 
experience  to  the  officers  in  the  S&DL  fields  cannot  start 
without  the  knowledge  of  the  tasks  performed  cy  these 
officers.  Seventy-five  percent  of  the  officers  in  tne  SSDF 
utilization  fields  provide  scientific,  technical,  and 
management  expertise  to  the  Air  Force  systems  acquisition 


community  (3:70).  The  AFSCs  associated  with  the  three  S&DE 
utilization  fields  are  26XX  Scientific  utilization  field, 
27XX  Acquisition  Program  Management  utilization  field,  and 
28XX  Development  Engineering  utilization  field. 

A  general  guide  to  the  professional  development  of  S&DE 
officers  is  provided  in  Air  Force  Regulation  36-23,  as  well 
as  the  following  general  descriptions  of  each  of  the  S&DE 
utilization  fields: 

AFSC  2 6XX  -  Scientific 

This  field  is  composed  of  many  different  scientific 
disciplines  grouped  together  because  of  commonality  of 
function--to  accomplish  basic  or  applied  research  under 
controlled  and  defined  conditions  to  produce  results 
that  will  add  to  the  pool  of  existing  scientific 
knowledge.  (3:69) 

AFSC  27XX  -  Acquisition  Program  Manager 

This  field  is  composed  of  officers  whe  work  the  diverse 
elements  associated  with  a  new  system's  entry  into  the 
operational  environment.  A  key  element  of  this  process 
is  the  delegation  by  the  Secretary  of  the  Air  Force 
(SAF)  of  its  executive  role  of  managing  all  the 
participating  commands  and  industrial  organizations  to 
the  system  program  manager  who  exercises  management 
through  a  system  program  office.  These  elements 
include  development,  test,  and  evaluation,  financial 
management,  configuration  management,  procurement, 
production,  logistics  support , training ,  and  site 
activation.  Officers  from  the  Scientific  (26XX), 
Development  Engineering  (28 XX),  and  other  areas  enter 
this  field  by  lateral  transfer  at  the  captain,  major, 
and  early  lieutenant  colonel  grades.  (3:69) 

AFSC  23XX  -  Development  Engineering 

This  field  includes  the  engineering  specialties 
concerned  with  the  designing,  developing,  installing, 
mod ifying,  testing,  and  analyzing  of  materials, 
techniques,  methods,  systems,  or  processes.  A 
significant  portion  of  the  activities  accomplished  in 
this  field  may  be  described  as  test  and  evaluation  and 
are  primarily  concentrated  in  AFOTEC  and  in  AFSCs  test 
centers,  test  ranges,  and  laboratories.  (3:09) 
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According  to  AFR  36-27  most  of  the  officers  in  S&DE 
fields  past  the  rank  of  major  will  eventually  move  into  the 
Acquisition  Management  (27XX)  career  field. 

Operational  Experience.  In  1985,  General  Lawrence 
Skantze,  then  Commander  of  Air  Force  Systems  Command, 
appointed  Major  General  Ronald  W.  Yates,  F-16  program 
director,  to  head  the  Acquisition  Management  Career 
Development  Task  Force  ( CDTF) .  The  task  force  composed  of 
twenty  senior  acquisition  managers  "acknowledged  external 
pressures  to  develop  a  better  career  development  path  for 
acquisition  managers,  and  agreed  that  improvements  in  their 
career  development  were  needed."  The  efforts  of  the  task 
force  resulted  in  the  implementation  of  the  Acquisition 
Management  Professional  Development  Program  (AMPDP)  (1:21). 

The  policies,  procedures  and  standards  of  the  AMPDP  are 
embodied  in  the  Air  Force  Systems  Command  Regulation  36-5. 
The  purpose  of  the  AMPDP  as  outlined  in  Air  Force  Systems 
Command  Reg  36-5  was: 

to  maximize  the  professional  development  and  mission 
capability  of  the  AM  officer  force  by  setting  forth  a 
definitive  and  viable  career  management  plan  that 
produces  broad-based  acquisition  managers  capable  of 
assuming  middle  management  and  senior  leadership  roles. 
(2:2) 

The  AMPDP  required  a  "broad  experience  in  several 
acquisition  disciplines  and  environments."  Therefore,  Air 
Force  Systems  Command  Regulation  36-5  provided  guidance 
concerning  the  type  of  assignments,  academic  education,  and 
professional  military  education  required  to  progress  through 
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the  AMPDP.  A  certification  process  was  established  that 
required  the  officer  to  progress  through  four  levels  of 
certification  prior  to  being  eligible  to  manage  a  major 
acquisition  program.  Furthermore,  Air  Force  Systems  Command 
Regulation  36-5  stated  that  "operational  experience  gained 
in  an  Air  Force  major  air  command  (other  than  AFSC  or  AFLC) 
or  in  a  joint  command  is  highly  desirable."  The  career 
areas  that  qualify  as  operational  tour  experience  are  listed 
in  Air  Force  Systems  Command  Regulation  36-5,  and  presented 
in  Table  l  (2:5). 


Table  1 

Operational  Utilization  Fields 


Utilization  Field 

Air  Force  j 

Specialty  Codes 

Pilot 

10  XX 

11XX 

12XX 

13  XX 

14  XX 

Navigator 

15  XX 

2  2  XX 

Air  Traffic  Control 

16  XX 

Air  Weapons  Controller 

17XX  | 

Missile  Operations 

1 

18  XX  ' 

Space  Operations 

A  y  y  !  1 

_ 20a\ _ ,, 

Weather 

2  5  X  X  || 

Missile  Maintenance 

3  1 XX  i 

Aircraft  Maintenance  S. 

4  0  XX  1 

Munitions 

Inte  1 1 iqence 

8  0  XX 

Security  Police 

8  1 X  X  ) 

A  I'  SC  A  _>  o  —  d 
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The  AMPDP  established  by  Air  Force  Systems  Command 
applied  to  all  the  utilization  fields  that  support  the 
acquisition  process.  Therefore,  it  provided  a  channel  for 
most  officers  in  acquisition  related  utilization  fields 
other  than  the  27XX  (Acquisition  Program  Management)  to 
develop  the  necessary  skill  to  assume  senior  acquisition 
management  positions. 

In  recognition  of  the  different  education  ana 
experience  requirements  of  the  different  acquisition  related 
utilization  fields,  the  Air  Force  developed  the  Acquisition 
Professional  Development  Prog  ran  (APDP)  with  different 
career  paths  for  each  utilization  field.  These  new  career 
paths  were  published  in  AFR  36-27,  in  26  December  1590.  The 
regulation  is  titled  Officer  Personnel:  Acquisition 
Professional  Development. 

The  requirements  for  certification  under  the  APDP  are 
similar  to  the  AMPDP,  but  they  are  tailored  to  fulfill  the 
requirements  of  each  acquisition  utilization  field. 

However,  AFR  36-27  establishes  a  certification  process  with 
three  levels  of  progression  instead  of  tour. 

The  stated  philosophy  of  the  APDP  is: 

(1)  Develop  acquisition  managers  with  a  common  basis  of 

education,  training  and  experience. 

(2)  Develop  acquisition  managers  'who  have  operational 

experience . 
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(3)  Provide  opportunities  for  professional  broadening 
in  related  acquisition  functional  areas. 

(4)  Provide  a  challenging  and  achievable  professional 
development  track. 

(5)  Be  understood  by  the  personnel  participating  in  the 
program  and  visible  to  the  public.  (9:7) 

As  stated  in  the  second  point  of  the  APDP  philosophy, 

operational  experience  is  an  important  aspect  of  the 

development  process.  Operational  experience  is  encouraged 

in  seven  of  the  eight  career  tracks  outlined  in  AFR  36-27. 

The  APDP  also  recognizes  the  fact  that  rated  officers  enter 

the  acquisition  career  fields  after  an  average  of  ten  years 

of  flying  due  to  time  constraints  imposed  by  the  Aviation 

Career  improvement  Act  (ACIA)  of  1933.  Therefore,  the  APDP 

provides  a  separate  career  track  for  rated  officers  (9:8). 

The  desirability  for  operational  experience  outlined  by 

the  AMPDP  and  later  by  the  APDP  was  validated  by  two 

research  studies.  On  September  1987,  Cap  Kevin  W .  Lopez, 

USAF,  completed  an  AFIT  thesis  titled  "Impact  of  AFSC 

Regulation  36-5  on  the  27XX  Career  Field."  In  his  study, 

Capt  Lopez  conducted  an  opinion  survey'  of  both  junior  and 

senior  officers  in  the  27XX  career  field  "to  determine  if 

the  provisions  cl  AFSC  Regulation  36-5  are  directly  related 

to  the  career  development  of  27XX  officers  (10:2)." 

Although  the  effects  of  operational  experience  on  yob 

performance  were  not  studied  by  Cap  Lopez,  his  study 

provided  insight  into  the  importance  ot  operational 

experience  to  career  progression  as  perceived  by'  acquisition 
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officers.  The  recipients  of  the  survey  developed  by  Capt 
Lopez  were  randomly  selected  from  all  the  officers  with  duty 
AFSCs  in  the  27XX  career  field  stationed  in  the  continental 
United  States.  The  number  of  officers  with  operational 
experience  who  returned  the  survey  was  not  reported. 

One  of  the  investigative  questions  addressed  the 
applicability  of  operational  experience  to  career 
development.  The  results  cf  the  survey  supported  the 
contribution  of  operational  experience;  approximately  63 
percent  of  the  junioi  officers,  and  75  percent  of  the  senior 
officers  responded  that  operational  experience  is  critical 
to  career  development  (10:49).  When  asked  about  the  minimum 
number  ct  years  of  operational  experience  necessary  for 
effective  ccr’ccir  d  o  v  g  1  c  p  it.  o  Pi  t  57  percent  of  tlio  ^ ur.ior 
officers  and  46  percent  of  the  senior  officers  responded 
three  years  or  less.  In  contrast,  21  percent  of  the  junior 
officers  and  40  percent  of  the  senior  officers  responded 
that  three  to  six  years  was  appropriate.  Despite  the  strong 
support  for  operational  experience,  19  percent  of  the  junior 
officers  and  10  percent  of  the  senior  officers  responded 
that  it  was  unnecessary  tor  career  development  (10:51). 

The  same  survey  used  by  Cap  Lopez  was  subsequently  used 
by  Capt  Reed  J.  McConnell  in  an  AFIT  thesis  research  ettort 
completed  in  Sep  ’OoS.  Between  the  time  that  Cap  hope.: 
released  the  survey  and  the  time  that  Cap  McConnell  started 
his  study .  t  in  ro.ee  Systems  Command  conducted  a  series  cl 
inform.  *x  u  briefings  to  educate  acquisition  oft  icons  on  the 


AMPDP.  Cap  McConnell  conducted  his  research  to  determine  if 
r  ,re  were  any  general  differences  in  responses  after  the 
education  effort  by  Air  Force  Systems  Command  was  completed. 
The  results  related  to  operational  experience  revealed  no 
changes  from  the  first  survey  (11:31). 

The  examination  of  Air  Force  Regulation  36-27  and  Air 
Force  Systems  Command  Regulation  36-5  only  revealed  the 
desirability  of  operational  experience  in  S&DE  utilization 
fields;  there  is  no  mention  of  specific  benefits  of 
operational  experience  in  these  regulations.  The  only 
regulation  which  mentions  specific  benefits  of  operational 
experience  in  these  fields  is  AFR  36-23. 

Specifically,  operational  experience  is  believed  to 
provide  the  following  benefits  to  officers  in  S&DE 
utilization  fields;  these  are  the  enhanced  ability  to 
interpret  operational  requirements,  enhanced  communicat ion 
with  the  users,  enhanced  credibility  for  personal  judgement, 
and  the  enhanced  ability  to  develop  useful  operator-oriented 
systems  (3:70)  . 

The  need  for  operational  experience  as  a  part  of  the 

career  development  of  acquisition  officers  is  supported  by 

high  ranking  Air  Force  officers.  In  relation  to  the  AMPDP, 

General  Bernard  P.  Randolph,  former  Commander  of  Air  Force 

Systems  Command  stated: 

since  there  is  no  substitute  for  operational 
experience,  we  look  for  flying  and  non-flying 
operations  too-missiles,  space,  and  munitions  or 
aircraft  maintenance.  Actually,  we  have  built  a 
professional  development  plan  that  accommodates 


15 


AMPDP .  Cap  McConnell  conducted  his  research  to  determine  if 
there  were  any  general  differences  in  responses  after  the 
education  effort  by  Air  Force  Systems  Command  was  completed. 
The  results  related  to  operational  experience  revealed  no 
changes  from  the  first  survey  (11:31). 

The  examination  of  Air  Force  Regulation  36-27  and  Air 
Force  Systems  Command  Regulation  36-5  only  revealed  the 
desirability  of  operational  experience  in  S&DE  utilization 
fields;  there  is  no  mention  of  specific  benefits  of 
operational  experience  in  these  regulations.  The  only 
regulation  which  mentions  specific  benefits  of  operational 
experier.ee  in  these  fields  is  AFR  36-23. 

Specifically,  operational  experience  is  believed  to 
provide  the  following  benefits  to  officers  in  S&DE 
utilization  fields;  these  are  the  enhanced  ability  to 
interpret  operational  requirements,  enhanced  communicat ion 
with  the  users,  enhanced  credibility  for  personal  judgement, 
and  the  enhanced  ability  to  develop  useful  operator-or i onte-d 
systems  (3:70)  . 

The  need  for  operational  experience  as  a  part  or  the 

career  development  of  acquisition  officers  is  supported  by 

high  ranking  Air  Force  officers.  In  relation  to  the  AM PUP, 

General  Bernard  P.  Randolph,  former  Commander  oi  Air  Force 

Systems  Command  stated: 

since  there  is  no  substitute  for  operational 
experience,  we  look  for  flying  and  non-living 
operations  too-niss l les ,  space,  and  munitions  or 
aircraft  maintenance.  Actually,  we  have  built  a 
professional  development  plan  that  accommodates 
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acquisition  officers,  test  pilots  and  navigators,  and 
operational  pilots  and  NAVS.  (12:6) 

Colonel  Michael  Butchko  stated  the  following  when  he 

was  the  program  director  of  the  F-15  fighter  aircraft  at 

Wright-Patterson  Air  Force  Base  in  Ohio: 

I'm  familiar  with  aviation,  which  I  think  is  vital  to 
someone  who  is  in  an  aircraft  acquisition  program.  If 
you  don't  speak  the  language,  you  don't  have  the 
credibility,  and  you  don't  know  how  to  ask  the  right 
questions.  (13:22) 

The  article  "A  Special  Breed  of  Cat:  The  Fighter  Pilch 

in  Systems  Acquisition  Management,"  by  Major  Lee  Lilly 

describes  the  successful  application  of  operational 

experience.  Major  Lilly  illustrates  how  a  pilot  discovered 

that  a  warning  device  developed  for  an  aircraft  was  so 

sensitive  that  it  detected  threats  too  far  away  to  present 

immediate  danger  to  the  pilot.  Since  the  warning  devi<  ? 

could  not  display  the  distance  to  the  threat,  a  close  threat 

was  treated  the  same  way  as  a  distant  threat.  For  all 

practical  purposes  warning  of  a  distant  threat  was 

equivalent  to  a  false  alarm  (14:34). 

Despite  the  support  for  operational  experience  as  an 

element  of  career  development  for  acquisition  officers, 

there  are  officers  in  the  Air  Force  who  beiie-.e  that 

operational  experience  is  not  necessarily  essential.  In  the 

article  "Buying  in  a  Fishbowl:  Two  Acquisition  Officers 

Comment,"  Lt  Col  Don  Ruths,  a  rated  officer  explained: 

There  are  all  sorts  of  ways  tc  gain  insight,  into  the 
operational  side  oi  the  house  without  actually  being 
tl  ire. . .  What's  important  is  to  develop  close 
relationships  with  your  prospective  customers.  (ID:  24) 
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In  1974,  J.  Ronald  Fox  a  former  Assistant  Secretary  of 


the  Army  reported  an  alternative  view  of  operational 
experience;  a  view  held  by  defense  contractors.  In  his  book 
Arming  America,  Fox  presents  the  comments  of  several 
presidents  and  vice  presidents  of  companies  with  major 
defense  and  space  contracts.  When  asked  for  an  opinion  on 
Government  personnel  managing  his  programs  one  executive 
expressed : 

The  military  man  who  has  operational  experience  need 
not  be  the  project  manager.  Indeed,  he  should  not  be 
the  project  manager  because  it  is  almost  impossible  for 
him  to  have  had  sufficient  training  and  experience  in 
managing  the  acquisition  process.  Rather,  the  military 
men  with  operational  experience  should  act  in  the 
cap,acity  of  consultants  to  the  project  manager. 

(16:189) 

A  second  industry  executive  expressed  a  similar 
opinion : 

The  Government  doesn't  need  technical  specialists  or 
operational  personnel  as  project  managers.  They  need 
business  managers  as  project,  managers  and  technical  and 
operational  people  as  consultants  to  the  project 
manager .  ( 16  :  189) 

The  critical  comments  provided  by  the  defense  industry 

executives  were  attributed  by  Fox  to  two  problems  that  faced 

officers  in  the  acquisition  community;  the  fast  rotation  of 

military  personnel  to  new  assignments;  and  the  lack  c i  a 

definite  career  progression  program  ior  acquisition 

personnel  (16:183,201).  Concerning  the  problem  of  military 

personnel  turnover  one  industry  executive  commented: 

The  change  ol  military  personnel  every  two  years  loads 
to  lack  of  responsibility  on  the  pa rt  of  both 
government  and  industry.  (16:183) 
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A  second  industry  executive  expressed  his  opinion  on 
the  adequacy  of  the  preparation  of  operational  personnel  in 
acquisition : 

Why  does  the  program  manager  or  top  guy  have  to  be  a 
pilot?  Industry  doesn't  run  their  projects  with 
pilots.  If  we  did,  we'd  be  out  of  business  in  no  time. 
(16:201) 

The  lack  of  career  programs  for  acquisition  officers 

affected  not  only  the  Air  Force,  but  the  other  services  as 

well.  One  industry  executive  commented  on  the  subject: 

There  should  be  career  progression  ladders  for 
procurement  officers  and  program  managers,  separate 
from  the  career  progression  ladders  for  operational 
personnel.  We  see  no  advantage  to  the  country  in 
taking  a  good  boat  commander  and  making  him  a  program 
manager.  (16:202) 

In  1974  when  Mr  Fox's  book  was  published,  the  Air  Force 
lacked  a  definite  career  progression  program  such  as  the 
AMPDP  and  the  APDP. 

Kerzner  provides  a  view  similar  to  that  of  the  industry 

executives  interviewed  by  Mr  Fox.  On  the  importance  cf 

business  skills  to  managers,  Kerzner  explains: 

The  program  manager  rarely  has  all  tne  technical, 
administrative,  and  marketing  expertise  needed  to 
direct  single-handedly.  Nor  is  it  necessary  or 
desirable.  It  is  essential,  however,  for  the  program 
manager  to  understand  the  technology,  the  markets,  and 
the  environment  of  the  business  to  participate 
effectively  in  hue  search  lor  integrated  solutions  and 
technological  innovations.  (16:170) 

Regarding  innovation,  Eric  Von  Hippel  argues  that  user 

experience  may  actually  be  undesirable.  In  the  article  "New 

Product  Ideas  From  'Lead  Users',"  professor  Hippel  states: 

users  selected  to  provide  input  data  to  consumer  and 
industrial  market  analyses  have  an  important 
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limitation:  Their  insights  into  new  product  needs  and 
potential  solutions  are  constrained  by  their  own 
experience.  Users  steeped  in  the  present  are  thus 
unlikely  to  generate  novel  product  concepts  that 
conflict  with  the  familiar. 

The  notion  that  familiarity  with  the  existing  product 
attributes  and  uses  interferes  with  an  individual's 
ability  to  conceive  of  novel  attributes  and  uses  is 
strongly  supported  by  research  into  problem-solving. 
(18:60) 

In  summary,  support  for  the  need  of  Air  Force 
acquisition  officers  with  operational  experience  varies. 
Operational  experience  is  officially  encouraged  in  the  Air 
Force.  However,  many  executives  in  industry  believe  that 
acquisition  officers  should  focus  their  career  development 
to  build  business  skills. 

Broadening  Experience  Tour  (BEST)  Program.  The  BEST 
program  was  created  by  Air  Force  Systems  Command  in  1984 
with  a  twofold  purpose;  to  provide  officers  in  S&DE  career 
fields  the  opportunity  to  obtain  operational  experience;  and 
to  provide  manning  to  critically  undermanned  operational 
career  fields.  The  key  elements  of  this  program  are  that  it 
is  a  voluntary  program  and  that  officers  are  guaranteed  they 
will  return  to  Air  Force  Systems  Command  at  the  end  of  the 
operational  tour. 

Once  a  year  Air  Force  Systems  Command  announces  the 
date  of  the  BEST  selection  board,  the  operational 
utilization  fields  that  are  critically  undermanned,  and  the 
eligibility  criteria.  The  eligibility  criteria  include  at 
least  three  years  time  on  station,  no  more  than  six  years  of 
commissioned  time,  availability  lor  reassignment,  volunteer 


status,  and  a  good  performance  record.  The  operational 
utilization  fields  available  for  fiscal  year  1992  include 
17 XX  (Air  Weapons  Controller) ,  18XX  (Missile  Operations) , 
20XX  (Space  Operations) ,  31XX  (Missile  Maintenance) ,  40XX 
(Aircraft  Maintenance),  and  80XX  (Intelligence)  (6). 

When  the  program  started  in  1984,  less  than  10  officers 
volunteered.  The  low  response  was  attributed  to  the  lack  of 
knowledge  of  the  new  program.  In  1985,  approximately  25 
engineers  volunteered  as  a  result  of  the  Engineering 
Crossflow  Program.  This  program  was  designed  to  reduce  the 
overmanned  20XX  career  field,  and  help  the  Air  Force 
Manpower  and  Personnel  Center  fill  other  undermanned  fields. 
As  a  result  of  the  Engineering  Crossflow  Program,  59 
officers  in  the  27XX  field  with  no  technical  education  were 
permanently  "crossf lowed"  to  other  career  fields;  and  75 
engineers  were  reassigned  between  the  BEST  program  and  the 
27 XX  field.  Since  1985  an  average  of  25  officers  have  been 
reassigned  every  year  through  the  BEST  program  (7). 

Conclus ions 

A  review  of  the  literature  describing  the  benefits  and 
applicability  of  operational  experience  in  the  acquisition 
process  revealed  a  clear  institutional  support  for  the  need 
of  this  type  of  experience.  The  benefits  and  desirability 
of  operational  experience  are  outlined  in  Air  Force 
regulations  and  supported  by  the  comments  of  Air  Force 
officials  and  opinion  surveys. 
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Despite  the  official  Air  Force  support,  a  different 
reaction  was  found  from  executives  of  DoD  and  space  related 
industries.  Industrial  executives  largely  disapproved  of 
the  placing  of  operational  personnel  in  acquisition  related 
jobs.  These  executives  favored  the  appointment  of  officers 
specialized  in  business  practices.  However,  it  was 
discovered  in  the  literature  that  the  reaction  of  these 
executives  was  published  several  years  prior  to  the 
implementation  of  the  Acquisition  Management  Professional 
Development  Program  in  the  late  80s,  and  the  Acquisition 
Professional  Development  Program  in  1990. 

Although  the  literature  available  provided  important 
insight  into  the  benefits  and  desirability  of  operational 
experience,  no  empirical  evidence  was  found  that  would 
confirm  that  operational  experience  enhances  the  ability  of 
S&DE  officers  to  develop  useful  operatcr-or iented  systems. 
Therefore,  primary  data  designed  to  study  the  effects  of 
operational  experience  had  to  be  collected.  The  details  of 
the  methodology  used  are  presented  in  chapter  III. 


2  1 


III.  Methodology 


Overview 

This  chapter  describes  the  methodology  used  to  collect 
and  analyze  the  primary  data  used  in  this  study. 

Specifically,  this  chapter  discusses  the  methodology  used  to 
examine  the  effects  that  operational  experience  may  provide 
to  S&DE  officers  to  develop  useful  operator-oriented 
systems . 

Research  Method 

A  cross-sectional  study  of  the  population  of  interest 
was  conducted  to  obtain  the  primary  data  needed  to  study  the 
effects  of  operational  experience. 

A  mail  survey  was  used  to  collect  this  data.  The  mail 
survey  was  chosen  for  its  versatility  and  economy  over  a 
telephone  survey. 

Sample/Population  of  Interest 

The  population  of  interest  for  this  research  was 
comprised  of  all  Air  Force  Systems  Command  officers  with 
duty  AFSCs  26XX  (Scientific  career  field),  27XX  (Acquisition 
Program  Management  career  field),  and,  23XX  (Development 
engineering  field).  To  answer  the  investigative  questions, 
a  minimum  of  two  samples  from  the  population  were  required; 
officers  who  attended  the  BEST  program  to  obtain  operational 
experience;  and  officers  in  the  S&DE  utilization  fields  with 
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no  operational  experience.  However,  samples  of  rated 
officers  assigned  to  S&DE  duties  and  non-rated  officers  who 
had  operational  experience  prior  to  entering  one  of  the  S&DE 
utilization  fields  were  also  drawn  to  study  possible 
differences  between  the  various  groups  of  officers  with 
operational  experience.  Therefore,  the  sampling  strategy 
used  was  a  stratified  sampling  of  officers  with  operational 
experience  and  those  without  it. 

Emory  states  that  "the  ideal  stratification  'would  be 
based  on  the  principal  variable  under  study  (19:307)."  The 
principal  variable  under  study  was  the  possession  of 
operational  experience.  The  reasons  for  using  stratified 
samples  when  the  population  can  be  segregated  into  subgroups 
are  "(1)  to  increase  a  sample's  statistical  efficiency;  (2) 
provide  adequate  data  for  analyzing  the  various 
subpopulations;  and  (3)  enable  different  research  methods 
and  procedures  to  be  used  in  different  strata  (19:307)." 

In  order  to  further  define  the  samples  and  reduce 
possible  variances  that  may  have  affected  the  results,  this 
study  considered  only  officers  with  active  duty  commissioned 
service  dates  between  1  September  1979  and  31  December  1934. 
These  dates  are  consistent  with  the  dates  of  active  duty 
commissioned  time  of  officers  who  nave  returned  to  Air  Force 
Systems  Command  from  BEST  assignments. 

Although  the  grade  spread  of  officers  in  S&DE 
utilization  fields  is  from  second  lieutenant  to  lieutenant 
colonel,  the  samples  included  only  captains  and  majors.  The 


limits  on  active  duty  commissioned  time  resulted  in  samples 
in  which  the  vast  majority  of  the  officers  were  captains. 

In  addition,  prior  enlisted  experience  in  operational  fields 
was  not  considered  since  prior  enlisted  assignment 
information  is  not  available  to  personnel  resource  managers 
at  the  time  assignments  are  made. 

In  addition  to  the  four  comparison  groups,  the 
supervisors  of  the  officers  selected  were  also  surveyed. 
Their  responses  were  used  to  provide  another  reference  for 
comparison . 

Sample  Size 

To  determine  the  size  of  the  samples  required,  a  data 
search  was  conducted  with  the  Air  Force  ATLAS  personnel 
computer  on  25  April  1991.  After  the  search  was  completed, 
the  determination  was  made  to  limit  sampling  to  the 
Aeronautical  Systems  Division  (ASD)  of  Air  Force  Systems 
Command  at  Wr ight-Patterson  Air  Force  Base,  and  to  conduct  a 
census  of  the  three  samples  of  officers  with  operational 
experience . 

The  sample  was  limited  to  ASD  for  two  reasons:  the 
number  of  officer's  that  had  participated  in  the  BEST 
program;  and  the  ability  to  obtain  the  name  and  addresses  of 
supervisors  from  the  2750th  ADW  Consolidated  Base  Personnel 
Office  and  the  ASD  personnel  office.  During  the  time  that 
this  study  was  conducted,  there  were  59  officers  who  had 
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participated  in  the  BEST  program  and  returned  to  Air  Force 
Systems  Command;  11  of  the  59  officers  were  assigned  to  ASD. 

The  census  of  the  remaining  officers  with  operational 
experience  included  16  rated  officers,  and  14  non-rated 
officers  who  were  "crossf lowed"  from  operational  assignments 
to  SiDE  assignments.  The  sample  of  officers  without 
operational  experience  consisted  of  51  officers  who  were 
selected  randomly  from  the  available  population.  The 
computer  search  of  officers  without  operational  experience 
considered  only  officers  without  prior  enlisted  experience. 
This  last  step  was  taken  to  assure  that  the  officers  indeed 
had  no  operational  experience. 

The  computer  search  for  the  information  on  the 
supervisors  revealed  that  some  supervisors  were  responsible 
for  more  than  one  of  the  surveyed  officers.  In  the  cases 
where  supervisors  were  responsible  for  more  than  one  officer 
surveyed,  these  supervisors  were  asked  to  answer  the  survey 
for  only  one  of  the  officers.  This  action  was  taken  to 
avoid  the  overburden  on  the  supervisor,  and  to  prevent  a 
high  rate  of  non-responses. 

Instrument 

General .  The  instrument  selected  for  data  collection 
was  a  mail  survey.  A  mail  survey  is  less  costly  than  a 
telephone  interview,  and  can  be  answered  by  the  respondent 
at  his  or  her  own  convenience.  In  addition,  it  allows  the 
respondent  to  remain  anonymous.  Anonymity  is  important. 


because  sensitive  performance  information  was  requested. 
Despite  the  advantages  of  the  mail  survey,  non-response 
remained  an  important  disadvantage  (19:172).  The  surveys 
used  are  provided  in  Appendix  A  and  Appendix  B. 

In  an  attempt  to  elicit  a  high  response  rate,  the 
survey  was  kept  as  short  as  possible.  Additionally,  no 
names  or  social  security  numbers  were  requested  to  preserve 
the  anonymity  of  the  respondents.  Further,  to  avoid 
possible  biases  that  could  result  from  possession  or  non¬ 
possession  of  operational  experience,  the  real  objective  of 
the  study  was  not  revealed  to  the  officers  surveyed. 
"Placebo"  questions  were  also  introduced  to  help  disguise 
the  real  objective  of  the  survey. 

Questionnaire  Structure.  The  survey  consisted  of  two 
parts;  the  first  part  collected  necessary  demographic 
information;  and  the  second  part  consisted  of  measurement 
questions . 

The  survey  requested  the  following  demographic 


information : 

1 . 

Respondent '  s 

rank 

2  . 

Respondent 1  s 

current  duty  AFSC 

3  . 

Respondent ' s 
of  years 

acquisition  experience,  in 

4  . 

Respondent 1  s 

aeronautical  rating 

5  . 

Respondent '  s 

operational  AFSC  other  than 

6  . 

Respondent ' s 
years 

operational  experience,  in 

number  or 


7.  Organization  where  the  respondent  was  currently 
assigned 

8.  Respondent's  primary  undergraduate  education 

9.  Respondent's  primary  graduate  education 

The  second  part  of  the  survey  consisted  of  two 

measuring  schemes  which  were  designed  to  answer  the 
investigative  questions.  Five  questions  were  created  for 
each  of  the  first  three  investigative  questions.  The 
questions  used  were  not  designed  to  measure  all  aspects  of 
every  purported  benefit  (enhance',  ability  to  interpret 
operational  requirements,  enhanced  communication  with  the 
using  organization,  and  enhanced  credibility).  Instead 
these  questions  were  designed  to  measure  in  broad  terms  the 
impact  of  operational  experience  on  each  purported  benefit. 
The  measurement  questions  were  designed  to  be  applicable  to 
officers  in  three  S&DE  career  fields.  Seven  "placebo" 
questions  were  also  introduced.  Although  the  purported 
benefits  of  operational  experience  are  supposed  to  apply  to 
systems  in  general,  the  questions  in  the  survey  were 
presented  in  relation  to  weapon  systems.  This  decision  was 
made,  because  ASD  primarily  acquires  aeronautical  "weapon" 
sys',  ^ns. 

The  second  measurement  scheme  required  respondents  to 
rank  six  skills  and  six  knowledge  areas.  The  purpose  of 
these  questions  was  to  determine  the  perceived  importance  c 
operational  experience  to  officers  who  participated  in  the 
BEST  program  and  officers  without  operational  experience. 


Officers  with  operational  experience  were  also  asked 


to  report  how  relevant  they  thought  their  operational 
experience  was  to  their  current  jobs.  The  intent  of  this 
question  was  to  answer  investigative  question  number  four. 
This  question  was  designed  to  determine  if  the  benefits  from 
operational  experience  are  dependent  on  the  relevancy  of 
specific  experience. 

The  answers  to  the  surveys  were  collected  on  machine 

sccreable  sheets.  All  the  answer  sheets  were  coded  to  allow 

identification  of  the  respondent's  sample  group. 

The  surveys  were  mailed  through  local  distribution  at 

Wright-Patterson  Air  Force  Base  on  July  1  1991.  A  suspense 

date  of  July  15  1991  was  given  to  return  the  completed 

surveys.  Once  the  responses  were  collected,  the  machine 

scoreable  sheets  were  optically  scanned. 

Instrument  Validity,  Reliability,  and  Practicality 

Emory  states  that  the  "characteristics  cf  sound  measurement" 

are  high  validity,  reliability  and  practicality  (19:94). 

Validity  refers  to  the  extent  to  which  a  test  measures 
what  we  actually  wish  to  measure.  Reliability  h  .s  to 
do  with  the  accuracy  and  precision  of  a  measurement 
procedure...  practicality  is  concerned  with  a  wide 
range  of  factors  of  economy,  convenience,  ar.d 
interpretabi 1 lty .  (19:94) 

Validity  can  be  categorized  as  internal  and  external. 
Internal  validity  is  defined  by  Emory  as  "the  ability  of  a 
research  instrument  to  measure  what  is  purported  to  measure 
(19:94)."  Fowler  describes  three  steps  to  improve  mternnx 
validity;  make  the-  questions  as  reliable  as  possible; 


include  enough  categories  of  responses  so  that  the  respondee 
can  provide  the  best  answer  possible;  and  include  various 
questions  with  different  formats  that  measure  the  same 
factor  (20:96).  To  increase  the  internal  validity,  five 
questions  were  provided  for  each  of  the  three  purported 
benefits  of  operational  experience  presented  in  chapter  I. 

In  addition,  the  survey  was  pretested  and  the  questions  were 
found  to  be  clearly  written  and  to  the  point.  However, 
limiting  the  number  of  questions  in  the  survey  presented  a 
trade-off  between  increasing  validity  and  increasing  the 
possible  number  of  expected  responses. 

External  validity  refers  to  the  ability  of  the 
instrument  to  measure  the  same  information  across  different 
groups  of  persons,  settings,  and  times  (19:94).  Although 
the  survey  instrument  was  provided  only  to  samples  of 
officers  with  six  to  11  years  of  active  duty  commissioned 
service,  the  survey  questions  were  written  with  the  complete 
population  in  mind.  However,  external  validity  was  not 
assessed . 

Reliability  refers  to  the  degree  that  the  survey 
provides  consistent  results.  According  to  Emory  reliability 
can  be  improved  if  external  variations  are  minimized 
(19:100).  Reliability  is  thus  increased  by  matching  the 
characteristics  of  the  sample  groups  as  much  as  possible. 

To  lower  variability,  the  number  of  years  of  commissioned 
service  was  controlled. 
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include  enough  categories  of  responses  so  that  the  respondee 
can  provide  the  best  answer  possible;  and  include  various 
questions  with  different  formats  that  measure  the  same 
factor  (20:96).  To  increase  the  internal  validity,  five 
questions  were  provided  for  each  of  the  three  purported 
benefits  of  operational  experience  presented  in  chapter  I. 

In  addition,  the  survey  was  pretested  and  the  questions  were 
found  to  be  clearly  written  and  to  the  point.  However, 
limiting  the  number  of  questions  in  the  survey  presented  a 
trade-off  between  increasing  validity  and  increasing  the 
possible  number  of  expected  responses. 

External  validity  reters  to  the  ability  of  the 
instrument  to  measure  the  same  information  across  different 
groups  of  persons,  settings,  and  times  (19:94).  Although 
the  survey  instrument  was  provided  only  to  samples  of 
officers  with  six  to  11  years  of  active  duty  commissioned 
service,  the  survey  questions  were  written  with  the  complete 
population  in  mind.  However,  external  validity  was  not 
assessed . 

Reliability  refers  to  the  degree  that  the  survey 
provides  consistent  results.  According  to  Emory  reliability 
can  be  improved  if  external  variations  are  minimised 
(19:100).  Reliability  is  thus  increased  by  matching  the 
characteristics  of  the  sample  groups  as  much  as  possible. 

To  lower  variability,  the  number  of  years  of  commissioned 


service  was  controlled. 


The  reliability  of  the  survey  was  measured  after 
collection  and  analysis  of  the  primary  data.  A  Cronbach 


alpha  was  selected  to  measure  reliability.  Cronbach ' s  alpha 
is  the  mean  of  all  possible  split-half  correlation 
coefficients  of  the  responses.  A  different  alpha  was 
computed  for  each  investigative  question  (21:196). 

A  problem  was  encountered  when  the  Cronbach  alphas  were 
computed.  The  measurement  scale,  which  is  discussed  in  the 
next  section,  was  not  structured  to  provide  equal  intervals. 
The  first  response  option  was  designed  for  officers  who 
ranked  themselves  in  the  lowest  40  percent  to  their  peers 
who  held  similar  jobs.  The  rest  of  the  response  options  in 
the  measurement  scale  increased  at  intervals  of  ten  percent. 
For  this  reason  the  Cronbach  alphas  were  computed  twice  (see 
Appendix  C) ;  once  with  the  measuring  scale  intact;  and  the 
second  time  with  the  first  item  deleted.  The  results  of 
both  computations  are  preser.ted  in  Table  2. 

Table  2 

Reliability  Measures  of  Survey  Questions,  by  Investigative 


Question  (Cronbach  Alphas) 


■ - — - - - — — — - 

I  r.vest . 

Invest . 

'I 

Invest.  i 

Question  1 

Question  2 

Question  3  j! 

Original  Seale 

.8  330 

.3393 

j! 

.  9  7.  3  *8 

1 

Modified  Scale 

.7269 

.  8998 

.9237 

These  results  suggest  the  questions  were  reliable  for 
use  in  this  study. 


Although  the  use  of  a  mail  survey  required  additional 
effort  to  obtain  authorization  from  the  Air  Force  Manpower 
and  Personnel  Center,  it  was  a  more  practical  instrument 
than  telephone  interviews.  The  survey  was  less  expensive, 
and  required  less  time  to  administer.  Additionally,  the 
survey  allowed  respondents  to  answer  when  it  was  most 
convenient  to  them. 

Finally,  the  survey  generates  standard  responses  so  it 
is  easier  for  a  person  not  involved  in  the  construction  of  a 
survey  to  interpret  the  results  (20:101). 

Selection  of  Measurement  Scale.  The  selection  of  a 
suitable  measuring  scale  is  critical.  The  measurement  scale 
determines  the  level  of  the  data  collected.  The  level  of 
data  in  turn  determines  the  type  of  statistical  tests  which 
are  appropriate  to  interpret  the  results  of  the  study.  In 
addition,  selection  of  a  measurement  scale  that  provides  a 
wide  choice  of  responses  can  increase  the  validity  of  the 
instrument  and  reduce  ambiguities. 

The  measurement  scale  used  in  this  survey  was  an 
ordinal  scale  which  allowed  respondents  to  select 
alternative  percentile  ratings.  Based  cn  the  results  cf  a 
pre-survey,  the  nuncer  of  response  options  was  reduced  from 
nine  to  seven.  Mon-parametric  statistical  tests  woie 
selected  since  this  response  scale  was  ordinal  and  the 
characteristics  of  the  population  distribution  were  unknown 


(  2  3  :  2  3)  . 


Statistical  Tests 


The  main  objective  of  this  study  was  to  examine  the 
effects  of  operational  experience  on  the  ability  of  S&DE 
officers  to  develop  useful  operator-oriented  systems.  This 
was  accomplished  by  measuring  the  difference  in  self 
reported  competency  between  samples  of  officers  with  and 
without  operational  experience.  The  statistical  tests 
applied  to  the  data  collected  in  this  study  were  selected 
for  their  ability  to  determine  whether  the  responses 
collected  with  the  survey  were  provided  by  samples  of 
officers  from  the  same  population. 

The  data  collected  with  the  individual  measurement 
question  is  of  the  ordinal  type,  and  the  median  is  the 
appropriate  measure  of  central  tendency  (20:89).  The  ’"edia 
test,  which  "is  a  procedure  for  testing  whether-  two 
independent  groups  differ  in  central  tendencies,"  was 
selected  (23:111).  If  a  significant  difference  was  not 
found,  the  implication  was  that  the  samples  came  from  the 
same  population.  However,  if  a  significant  difference  was 
found,  then  the  implication  was  that  samples  were  drawn  fro 
different  populations.  Tests  between  samples  were  made  for 
each  question  (i.e.  BEST  officers  versus  officers  with  no 
operational  experience,  etc.).  The  results  of  all  the  test 
are  provided  in  Appendix  D  and  the  significant  results 
discussed  in  chapter  IV. 

The  Friedman  two-way  analysis  of  variance  was  used  to 
test  the  ranking  schemes.  This  test  is  used  to  evaluate  1: 
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k  number  of  samples  are  drawn  from  the  same  population 
(22:166).  The  Friedman  statistic  computed  with  this  test 
was  used  to  compute  W,  the  Kendall's  coefficient  of 
concordance.  The  coefficient  of  concordance  W  is  a 
measurement  of  agreement  between  the  survey  respondents. 

This  test  is  typically  used  to  assess  the  level  of  agreement 
among  judges.  The  coefficient  of  concordance  is  linearly 
related  to  the  Friedman  statistic  by  W  =  X/n(r-l) ,  where  n 
is  the  number  of  samples  and  r  is  the  number  of  ranked  items 
(22:229).  In  addition  to  the  Friedman  test,  a  multiple 
pairwise  testing  procedure  introduced  by  Netter,  Wasserman 
and  Kutner  was  used.  This  p  euure  was  used  to  test  the 
comparative  magnitude  of  the  means  of  the  ranked  items. 

This  last  procedure  is  analogous  to  the  "Bonferroni  pairwise 
comparison"  (23:950).  The  results  of  the  Friedman  test  and 
the  multiple  pairwise  comparisons  are  presented  in 
Appendix  E. 

Finally  Spearman  Rank  Correlation  coefficients  were 
computed  for  all  questions  to  determine  the  strength  of 
association  between  supervisor  and  subordinate  responses 
(22:202)  . 

The  statistical  tests  and  analyses  were  conducted  using 
the  statistical  computer  programs  Statistix  3.1  running  on  a 
personal  computer,  and  SAS  6.0  running  on  a  mainframe 


computer . 


Summary 

This  chapter  provided  all  the  information  pertinent  to 
the  methodology  applied  in  the  collection  and  analysis  of 
the  primary  data.  The  samples  from  the  population  of 
interest  were  defined  as  well  as  the  data  collection 
instrument.  The  appropriate  statistical  tests  as  well  as 
the  limitations  and  assumptions  were  also  identified.  The 
results  of  the  data  primary  collection  effort  and  analysis 


are  presented  in  chapter  IV. 


Introduction 

This  chapter  describes  the  results  of  the  analysis 
conducted  on  the  data  collected  with  the  survey.  The 
description  of  the  response  to  the  survey  is  presented  first 
followed  by  the  demographic  statistics  collected.  Finally, 
the  specific  findings  and  analysis  for  all  survey  questions 
are  presented. 

Survey  Response 

The  survey  response  rates  for  all  samples  of 
survey  recipients  are  listed  in  Table  3. 


Table  3 

Survey  Response  Rate  by  Sample 
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The  overall  response  rate  was  47.6%.  The  response  rate 
for  the  supervisors  was  lower  than  the  rate  for  the  officers 
studied . 

Demographic  Statistics 

The  first  part  of  the  survey  collected  information 
required  to  present  vital  characteristics  of  the  various 
groups  of  officers  studied.  The  first  demographic  data 
presented  is  the  number  of  officers  in  each  S&DE  career 
field.  This  data  is  presented  in  Table  4. 

Table  4 


Composition  o f  Samples  by  S&DE  Utilization  Fields 


Scientists 

Acquisition 
Program  Mgrs 

Engineers 

GROUP 

2  6XX 

27XX 

2  S  XX 

BEST  Officers 

1 

/ 

Rated  Officers 

8 

2 

Other-ops 

1 

5 

1 

Non-ops  Officers 

1 

5 

15 

TOTAL 

2 

19 

25 

Percentage 

4.4% 

4  1.3% 

54  .  3% 

The  majority  (54.3%)  of  the  officers  surveyed  had  a 
duty  AFSC  of  2 8 XX  (Development  Engineering).  The 
preponderance  of  the  remaining  officers  were  (42.3%) 
Acquisition  Program  Managers,  AFSC  27XX.  There  were  only 
two  scientists,  AFSC  26XX,  who  responded  to  the  study 
survey.  Seven  of  the  8  BEST  program  participants  were 
engineers.  Only  one  participant  in  the  BEST  sample  was  an 
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acquisition  program  manager.  The  vast  majority  of  officers 
in  the  sample  without  operational  experience  were  also 
engineers.  On  the  other  hand,  the  majority  of  the  rated 
officers  as  well  as  the  non-rated  officers  who  acquired 
their  operational  experience  before  assignment  to  S&DE 
positions  held  acquisition  program  management  positions. 

Another  important  demographic  factor  examined  was  the 
number  of  years  of  acquisition  experience  possessed  by  the 
officers  surveyed.  Acquisition  experience  was  defined  in 
the  survey  as  experience  gained  from  working  in  any  of  the 
S&DE  career  fields.  The  number  of  years  of  acquisition 
experience  are  summarized  by  samples  in  Table  5. 


Table  5 


possessed  the  most  acquisition  experience,  followed  by  the 
officers  in  the  BEST  sample.  The  sample  with  the  least 
acquisition  experience  was  the  sample  of  rated  officers. 
The  majority  of  the  rated  officers  surveyed  possessed  less 
than  two  years  of  acquisition  experience. 
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The  demographic  statistics  collected  also  provide 
information  regarding  the  operational  specialties  of  the 
officers  in  the  various  samples.  The  sample  of  rated 
officers  was  composed  of  four  pilots  and  six  navigators. 
The  numbers  of  officers  possessing  the  various  non-rated 
operational  specialties  are  provided  in  Table  6. 


Table  6 

Composition  of  Samples  by  Non-rated 
Operational  Utilization  Fields 
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The  key  finding  from  Table  6  is  that  all  of  the  officers 
in  ASD  who  participated  in  the  BEST  program  acquired 
operational  experience  as  aircraft  maintenance  officers. 

The  number  of  years  of  operational  experience  was  also 
collected.  This  information  is  presented  in  Table  7.  The 
majority  of  the  BEST  officers  surveyed  possessed  two  to  four 
years  of  operational  experience,  while  the  majority  of  the 
rated  officers  had  six  to  ten  years  of  operational 
experience.  The  majority  of  the  officers  who  entered  the 
S&DE  field  after  non-rated  operational  assignments  had  four 
to  six  years  of  operational  experience.  Since  rated 
officers  have  to  undergo  extensive  training  for  a  long 
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period  of  time,  rated  officers  are  not  allowed  to  transfer 
to  S&DE  duties  as  early  as  officers  with  other  non-rated 
operational  experience. 


Table  7 


In  summary,  the  demographic  statistics  provided  a 
detailed  view  of  the  various  characteristics  of  the 
different  samples.  Seven  out  of  eight  of  the  BEST  officers 
were  engineers  and  all  of  them  had  experience  as  aircraft 
maintenance  officers.  The  majority  of  the  officers  without 
operational  experience  were  also  engineers.  In  contrast, 
the  majority  of  the  rated  officers  who  participated  in  this 
study  were  acquisition  program  officers. 

Specific  Findings  and  Analysis 

This  section  presents  the  results  of  the  analysis 
conducted  in  this  study.  The  findings  from  each  individual 
survey  question  are  presented  along  with  the  relevant 
investigative  question. 
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Investigative  Question  1 

investigative  question  number  one  explores  whether  the 
operational  experience  acquired  through  the  BEST  program 
enhances  the  ability  oi  S&DE  officers  to  interpret 
operational  requirements.  Survey  questions  number  11,  14, 
19,  22,  and  26  were  designed  to  answer  investigative 

question  number  one.  The  results  from  each  question  are 
presented  below: 

Survey  Question  11.  When  it  comes  to  evaluating  the 
logistics  support  required  to  maintain  the  weapon  system(s)/ 
component(s)  that  my  organization  develops/produces,  I  do 
better  than  _  of  my  peers  whose  jobs  are  similar  to  mine. 

The  median  responses  from  all  samples  for  question  11  are 
presented  in  Figure  l. 
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Figure  1.  Median  Responses  for  Survey  Question  11 
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Question  11  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to  evaluate 


logistics  support.  The  self  reported  ability  of  officers 
from  the  BEST  sample  to  evaluate  logistics  support  was 
higher  (at  a=.05)  than  the  self  reported  ability  of  officers 
from  all  other  samples.  There  was  no  statistical  difference 
based  on  supervisor  ratings. 

Survey  Question  14.  When  it  comes  to  reading  and 
comprehending  the  operational  deficiencies  outlined  in 
operational  requirements  documents  such  as  the  Statement  of 
Need  (SON) ,  the  Statement  of  Operational  Requirements 
Document  (SORD) ,  the  Operational  Requirements  Document 
(ORD) ,  and  the  Mission  Need  Statement  (MNS) , 

I  do  better  than  _  of  my  peers  whose  jobs  are  similar  to 

mine . 

The  results  for  question  14  are  presented  in  Figure  2. 


Figure  2. 


Median  Responses  for  Survey  Question 


Question  14  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
understand  operational  requirements  as  they  are  officially 
documented.  The  self  reported  ability  of  rated  officers  to 
understand  deficiencies  outlined  in  operational  requirements 
documents  was  higher  (at  a-. 05)  than  that  of  officers  with 
no  operational  experience.  There  was  no  statistical 
difference  based  on  supervisor  ratings. 

Survey  Question  19.  when  evaluating  alternative 
technical  solutions  to  the  operational  deficiencies  which 
will  be  corrected  by  the  weapon  system ( s ) /component ( s )  that 

my  organization  develops/produces,  I  do  better  than _ of  my 

peers  whose  jobs  are  similar  to  mine. 

The  results  for  question  19  are  presented  in  Figure  3. 


Question  19  was  included  to  determine  if  operational 


experience  enhances  the  ability  of  the  S&DE  officers  to 
evaluate  technical  solutions  to  operational  deficiencies. 

The  self  reported  ability  of  officers  from  the  BEST  sample 
to  evaluate  technical  solutions  to  operational  deficiencies 
was  higher  (at  a=.05)  than  that  of  officers  with  other  non- 
rated  operational  experience.  There  was  no  statistical 
difference  based  on  supervisor  ratings. 

Survey  Question  When  assessing  the  appropriateness 

of  performance  parameters  in  system  level  specifications  and 

other  technical  documents,  I  do  better  than  _  of  my  peers 

whose  jobs  are  similar  to  mine. 

The  results  for  question  22  are  presented  in  Figure  4. 


Figure  4.  Median  Responses  ler  Survey  Question  : 


Question  22  was  included  to  determine  if  operational 


experience  enhances  the  ability  of  the  S&DE  officers  to 
assess  performance  parameters  of  weapon  systems.  No 
significant  statistical  difference  (at  a=.05)  in  self 
reported  ability  to  assess  the  appropriateness  of 
performance  parameters  outlined  in  system  level 
specifications  and  other  technical  documents  was  detected 
between  any  of  the  samples  of  officers  surveyed.  There  was 
no  statistical  difference  based  on  supervisor  ratings. 

Survey  Question  26.  When  evaluating  the  test 
requirements  needed  to  assess  the  military  worthiness  of  the 
weapon  system (s)/  component(s)  that  my  organization 

develops/produces,  I  do  better  than  _  of  my  peers  whose 

jobs  are  similar  to  mine. 

The  results  for  question  26  are  presented  in  Figure  5. 


Question  26  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to  evaluate 


test  requirements. 

The  self  reported  ability  of  rated  officers  to  evaluate 
test  requirements  needed  to  assess  the  military  worthiness 
of  weapons  systems/components  was  higher  (at  a=.05)  than 
that  of  officers  with  no  operational  experience  as  well  as 
those  with  other  non-rated  operational  experience.  The  self 
reported  ability  of  officers  from  the  BEST  sample  was  higher 
(at  q=.0 -j>)  than  that  of  officers  with  other  non-rated 
operational  experience.  A  difference  (at  a=.05)  was 
detected  in  the  rating  of  the  supervisors  of  officers  with 
no  operational  experience  who  ranked  their  subordinates 
higher  than  the  supervisors  of  the  officers  in  the  BEST 
sample . 

In  summary,  officers  who  participated  in  the  BEST 
program  do  not  report  an  enhanced  ability  to  interpret 
operational  requirements  over  those  officers  with  no 
operational  experience. 

Investigative  Question  2 

Investigative  question  number  two  explores  whether 
operational  experience  acquired  through  the  BEST  program 
enhances  the  ability  of  S&DE  officers  to  ccnaunicate  with 
using  organizations.  Survey  questions  13,  17,  2-*.,  27,  and 


31  were  designed  to  answer  investigative  question  number 
two.  The  results  from  each  question  are  presented  below: 


Survey  Question  13.  When  it  comes  to  briefing  the 
users  of  the  weapon  system(s)/component (s)  that  my 

organization  develops/produces,  I  do  better  than  _  of  my 

peers  whose  jobs  are  similar  to  mine. 

Question  13  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
communicate  with  using  organizations  through  formal 
presentations.  The  median  responses  from  all  samples  for 
question  13  are  presented  in  Figure  6. 


ABILITY  TO  BRiEF  the  users 
OF  WEAPON  SYSTEMS 
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Figure  6.  Median  Responses  for  Survey  Question  13 

No  significant  statistical  difference  (at  a-. OS)  was 
found  between  the  self  reported  ability  of  of fleers  surveyed 
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to  brief  the  users  of  weapon  systems.  There  was  no 
statistical  difference  based  on  supervisor  ratings. 


Survey  Question  17.  When  it  comes  to  understanding  the 
language  used  in  operational  requirements  documents  such  as 
the  Statement  of  Need  (SON) ,  the  Statement  of  Operational 
Requirements  Document  (SORD) ,  Operational  Requirements 
Document  (ORD) ,  and  the  Mission  Need  Statement  (MNS),  I  do 
better  than  _  of  my  peers  whose  jobs  are  similar  to  mine. 

Question  17  was  included  tc  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
understand  the  requirements  ,of  the  operational  users  as 
officially  documented.  The  median  responses  from  all 
samples  for  question  17  are  presented  in  Figure  7. 
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Figure  7.  Median  Responses  for  Survey  Question  17 

The  self  reported  ability  of  rated  officers  to 
understand  the  language  used  in  operational  requirements 


documents  was  higher  (at  a=.05)  than  that  of  officers  with 
no  operational  experience.  There  was  no  statistical 
difference  based  on  supervisor  ratings. 

Survey  Question  24.  When  it  comes  to  understanding  the 
acronyms  used  by  the  users  and  maintainers  of  the  weapon 
system(s)/  component (s)  that  my  organization 

develops/produces,  I  do  better  than  _  of  my  peers  'whose 

jobs  are  similar  to  mine. 

Question  24  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
understand  the  acronyms  used  by  users  and  maintainers  of 
weapon  systems.  The  median  responses  from  all  samples  for 
question  24  are  presented  in  Figure  3 . 


Figure  3.  Median  Responses  for  Survey  Question  24 
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The  self  reported  ability  of  officers  from  the  BEST 
sample  to  understand  the  acronyms  used  by  users  and 
maintainers  of  weapon  systems  was  higher  (at  a=.05)  than 
that  of  officers  with  no  operational  experience  and  officers 
with  other  non-rated  operational  experience.  There  was  no 
statistical  difference  based  on  supervisor  ratings. 

Survey  Question  27.  When  it  comes  to  determining  the 
organization  in  the  using  command  which  can  provide  the 
various  types  of  data  required  to  do  my  job,  I  do  better 
than  _  of  my  peers  whose  jobs  are  similar  to  mine. 

The  median  responses  from  all  samples  for  question  27 
are  presented  in  Figure  9. 


Figure  9.  Median  Responses  for  Survey  Question  27 


Question  27  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  D£.DE  officers  to 
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determine  where  to  seek  information  within  the  using 
organization.  The  self  reported  ability  of  officers  from 
the  BEST  sample  to  determine  the  organization  in  the  using 
command  which  can  provide  the  various  types  of  data  required 
to  do  their  work  was  higher  (at  a=.05)  than  that  of  officers 
with  no  operational  experience.  There  was  no  statistical 
difference  based  on  supervisor  ratings. 

Survey  Question  31.  When  determining  the  type  of  data 
required  by  maintenance  personnel  to  support  the  weapon 
system (s)/  component (s)  that  my  organization 

develops/produces,  I  do  better  than  _  of  my  peers  whose 

jobs  are  similar  to  mine. 

The  results  for  question  31  are  presented  in  Figure  10. 


ABILITY  TO  DETERMINE  DATA  REOUIRlD  By 
MAINTAINERS  OF  WEAPON  SYSTEMS 
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Figure  10.  Median  Responses  for  Survey  Question  31 
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Question  31  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
determine  the  type  of  data  required  by  maintenance 
personnel.  Part  of  being  able  to  communicate  with  the  user 
is  knowing  the  type  of  information  needed  by  maintenance 
personnel . 

The  self  reported  ability  of  officers  from  the  BEST 
sample  to  determine  the  type  of  data  required  by  maintenance 
personnel  to  support  weapon  systems  was  higher  (at  a=.05) 
than  the  self  reported  ability  of  all  other  officers.  There 
was  no  statistical  difference  based  on  supervisor  ratings. 

In  summary,  the  self  rating  of  officers  who 
participated  in  the  BEST  program  reflect  a  greater  ability 
to  communicate  with  the  using  organization  than  officers  in 
other  samples.  This  ability  is  most  evident  regarding 
familiarity  with  maintenance  data  requirements,  acronyms, 
and  sources  of  information. 

Investigative  Question  3 

Investigative  question  number  three  explores  whether 
the  operational  experience  acquired  through  the  BEST  program 
helps  S&DE  officers  establish  credibility  for  their 
judgment.  Survey  questions  10,  12,  20,  30,  and  32  were 

designed  to  answer  investigative  question  number  three.  The 
questions  measure  their  ability  to  influence  development- 
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production  decisions,  and  decisions  affecting  capabilities 
of  weapon  systems.  The  results  from  each  questions  are 
presented  below: 

Survey  Question  10.  When  it  comes  to  providing  advice 
on  matters  affecting  the  development/production  of  the 
weapon  system ( s) /component ( s)  in  my  organization,  I  am 

called  upon  more  often  than  _  of  my  peers  whose  jobs  are 

similar  to  mine. 

Question  10  was  included  to  determine  if  operational 
experience  increases  the  frequency  that  S&DE  officers  are 
called  upon  to  provide  advice  on  matters  affecting  the 
develcpment/product ion  of  weapon  systems.  The  median 
responses  from  all  samples  for  question  10  are  presented  in 
Figure  11. 


Figure  11.  Median  Responses  for  Survey  Question  10 
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No  statistical  difference  (at  a=.05)  was  detected  in 


the  self  reported  frequency  of  officers  sampled  when  called 
upon  to  provide  their  advice  on  matters  affecting  the 
development/production  of  weapon  system(s) /component (s) . 
However,  it  is  worth  noting  that  the  median  score  for  the 
sample  of  officers  with  no  operational  experience  was  the 
highest  at  70%  percent,  and  the  same  score  was  reported  by 
their  supervisors.  There  was  no  statistical  difference 
based  on  supervisor  ratings. 

Survey  Question  12.  When  it  comes  to  influencing 
decisions  affecting  the  capabilities  of  the  weapon 
system ( s) /component (s)  that  my  organization  develops/ 

produces,  I  do  better  than  _  of  my  peers  whose  jobs  are 

similar  to  mine. 

The  results  for  question  12  are  presented  in  Figure  12 
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Figure  12.  Median  Responses  for  Survey  Question  12 


Question  12  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
influence  decisions  affecting  the  capabilities  of  weapon 
systems.  No  statistical  difference  (at  a=.05)  was  detected 
in  the  self  reported  ability  of  officers  sampled  to 
influence  decisions  affecting  the  capabilities  of  weapon 
systems.  However,  it  is  worth  noting  that  the  median  score 
of  rated  officers  was  the  highest  at  80  percent.  There  was 
no  statistical  difference  based  on  supervisor  ratings. 

Survey  Question  20.  when  it  comes  to  providing  advice 
on  matters  affecting  the  capabilities  of  the  weapon 
system ( s) /component ( s )  that  my  organization  develops/ 

produces,  I  am  called  upon  more  often  than  _  of  my  peers 

whose  jobs  are  similar  to  mine. 

The  iesuics  tor  question  20  are  presented  in  Figure  13. 


Figure  13.  Median  Responses  lor  Survey  Question  20 
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Question  20  was  included  to  determine  if  operational 
experience  enhances  the  opportunities  of  S&DE  officers  to 
provide  advice  on  matters  affecting  the  capabilities  of 
weapon  system(s)/component(s) . 

No  statistical  difference  (at  a=.05)  was  detected  in 
the  self  reported  opportunities  of  officers  sampled  to 
provide  their  advice  on  matters  affecting  the  capabilities 
of  weapon  systems.  There  was  no  statistical  difference 
based  on  supervisor  ratings. 

Survey  Question  30.  When  it  comes  to  providing  my 
opini/on  on  decisions  affecting  the  capabilities  of  the 
weapon  system ( s) /component ( s )  that  my  organization 

develops/produces,  I  am  called  upon  more  often  than  _  of 

ny  peers  whose  jobs  are  similar  to  mine. 

The  results  for  question  30  are  presented  in  Figure  14. 
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Question  30  was  included  to  determine  if  operational 
experience  enhances  the  opportunities  of  S&DE  officers  to 
provide  their  opinion  on  decisions  affecting  the 
capabilities  of  weapon  system ( s ) /component ( s) . 

No  statistical  difference  (at  a=.05)  was  detected  in 
the  self  reported  opportunities  of  officers  sampled  to 
provide  their  opinion  on  decisions  affecting  the 
capabilities  of  weapon  systems.  However,  it  is  worth  noting 
that  the  median  score  of  rated  officers  was  the  highest  (at 
70  percent)  between  the  officers  sampled.  In  addition,  the 
median  score  reported  by  supervisors  of  officers  with  other 
non-rated  operational  experience  was  80  percent.  There  was 
no  statistical  difference  based  on  supervisor  ratings. 

Survey  Question  32.  When  it  comes  to  influencing  the 
decisions  affecting  the  development/production  of  the  weapon 
system ( s ) /component ( s )  in  my  organization,  I  do  better  than 
_  of  my  peers  whose  jobs  are  similar  to  mine. 

Question  32  was  included  to  determine  if  operational 
experience  enhances  the  ability  of  S&DE  officers  to 
influence  decisions  affecting  the  develcpment/product ion  of 
weapon  system(s)/  component ( s) .  The  median  responses  from 
all  sample0  for  question  32  are  presented  in  Figure  15. 

The  self  reported  ability  of  rated  officers  to 
influence  decisions  affecting  the  development/production  of 
weapon  systems/components  was  higher  (at  a=.05)  than  that  c: 
officers  witn  no  operational  experience.  There  was  no 
statistical  difference  based  on  supervisor  ratings. 
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Figure  15.  Median  Responses  for  Survey  Question  32 


In  summary,  there  was  no  practical  difference  in  the 
level  of  credibility  for  their  judgment  between  all  samples 
of  officers  surveyed. 

Investigative  Question  4 

Investigative  question  four  explores  hew  the  relevancy 
of  specific  operational  experience  affects  the  ability  of 
S&DE  officers  to  develop  useful  operator-oriented  systems. 
Question  45  was  used  to  help  answer  investigative  question 
four.  Question  45  was  designed  to  divide  tne  samples  of 
officers  with  operational  experience  into  groups  of  officers 
possessing  relevant  operational  experience,  and  officers 


possessing  irrelevant  operational  experience.  Once  divided, 
the  results  of  the  measurement  questions  for  investigative 
questions  one  through  four  were  to  be  analyzed  and  reported. 
However,  this  analysis  was  not  possible  because  only  two 
officers  indicated  their  operational  experience  was 
irrelevant.  The  results  from  the  supervisors  could  not  be 
used  either,  because  they  were  not  asked  to  report  whether 
the  operational  experience  possessed  by  their  subordinates 
was  relevant  to  their  jobs  or  not. 

Measurement  of  Agreement:  Supervisors  and  Subordinates 

When  the  survey  data  was  collected,  ~'^e  coded  numbers 
cn  the  answer  sheets  were  used  to  identify  matched  pairs  of 
supervisors  and  subordinates;  eignteen  pairs  were  obtained. 
The  responses  from  each  matched  pair  were  used  to  obtain 
Spearman  rank  correlations  for  each  of  the  15  measurement 
questions.  These  correlations  represent  a  measure  of 
agreement  between  the  way  that  supervisors  and  subordinates 
answered  questions. 

The  results  of  the  Spearman  rank  correlation  were 
corns  1  stent  with  the  results  from  the  analysis  conducted  on 
the  15  measurement  questions;  namely,  in  general  there  was 
little  agreement.  The  computed  Spearman  rank  correlation 
coefficients  are  presented  in  Table  8. 


skills.  The  results  of  the  rankings  for  each  sample  of 
officers  is  presented  in  Table  9.  The  supervisors  were  not 
asked  to  participate  in  the  rankings. 


Table  9 

Ranking  of  Skills  by  Samples 


BEST 

- 

No  Operational  Exp. 

1 

Decision  Making 

H 

Communication 

2 

Operational 

2 

Technical 

3 

Communication 

3 

Decision  Making 

*■* 

Technical 

H 

Leadership  i 

5 

Leadership 

5 

Operational  j 

6 

Administrative 

6 

Administrative  : 

W  -.291,  sig  at  a =.05 

W  =.43,  s iq  at  a= . 05  P 

Rated 

Other-Ops 

1 

Communication 

1 

Leadership  | 

k 

Decision  Making 

2 

Decision  Making  ; 

* 

Leadersh ip 

k 

1 

Technical  1 

'1 

Ooerat ionai 

m 

Communication 

5 

Technical 

n 

Operational 

6 

Administrative 

^7 

•  ■  ! 

Administrative 

■ 

i 

W  =.34,  not  stat  sig  1 

i - 

*  Tied  Rankings 

1 

The  results  at  the  ranking  and  statistical  tests 
indicate  that  oificers  L  rein  the  BEST  sample  foci  that 
operational  skills  are  more  important  than  officers  with  r.o 
operational  experience ,  who  ranked  operational  skills  lower. 

The  iirst  tost,  conducted  measures  the  comparative 
magnitude  ol  the  ranked  skills  (see  Append i x  K) .  The 


on 


results  indicate  that  the  officers  in  the  BEST  sample 
strongly  felt  (a=.l)  that  decision  making  and  operational 
skills  are  more  important  than  administrative  skills. 

However,  the  officers  with  no  operational  experience 
strongly  felt  (a~.l)  that  communication ,  technical,  decision 
making,  and  leadership  skills  are  more  important  than 
administrative  skills.  In  addition  officers  with  no 
operational  experience  strongly  felt  (a=.l)  that 
co;  imunicat  ion  skills  are  more  important  than  operational 
skills . 

The  coefficient  of  concordance  W,  which  is  a  measure  of 
agreement  between  the  officers  in  each  sample,  was  computed 
fcr  all  samples.  When  W  is  statistically  significant,  it 
means  that  the  agreement  between  the  officers  in  a  sample  is 
higher  than  it  would  be  by  chance  (22:237).  The  agreement 
between  officers  with  no  operational  experience  was  stronger 
than  that  of  the  officers  from  the  BEST  sample. 

The  rated  officers  and  other  non-rated  officers  with 
operational  experience  were  more  varied  in  their  responses 
than  officers  in  the  BEST  sample.  This  is  indicated  by  the 
ties  in  the  ranking  of  skills  presented  in  Table  9.  ..'hat 
the  ties  represent  is  that  these  oft icers  did  not  reel 
strongly  about  the  ranking  of  the  tied  skills. 

Ranking  of  Knowledge  Areas 

The  respondents  also  ranked  a  list  of  knowledge  areas, 
two  ct  which  were  tactics  ot  deployment  and  xa  i  :i  tor.  unco 
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required  t.o  keep  weapon  systems  working.  The  intent,  of  this 
measurement  scheme  was  not  to  determine  if  operational 
experience  enhances  the  ability  of  UaDE  officers  to  develop 
useful  operator-oriented  systems,  but  to  investigate  the 
relative  importance  of  operational  knowledge  to  the  officer 
surveyed.  The  knowledge  areas  representing  operational 
knowledge  were  maintenance  and  tactics  of  deployment.  The 
other  ranked  areas  were  technology,  threat,  acquisition 
process,  and  acquisition  regulations.  The  result  of  the 
rankings  of  knowledge  areas  for  each  sample  of  officers  is 
presented  in  Table  10.  The  supervisors  were  not  asked  to 
participate  in  the  rankings. 

The  results  of  the  rankings  and  statistical  tests 
indicate  that  the  officers  from  the  BEST  sample  and  the 
officers  with  no  operational  experience  ranked  the  knowledge 
areas  in  the  same  manner.  The  coefficients  of  concordance 
were  high  for  both  samples,  and  statistically  significant  at 
a=. 05. 

The  tests  for  the  comparative  magnitude  of  the  ranked 
knowledge  areas  indicated  that  officers  from  the  BEST  sample 
strongly  felt  (a=.l)  that  the  areas  of  knowledge  of 
technology  tc  make  the  weapon  system  work,  tactics  of 
deployment,  and  maintenance  were  more  important  than 
knowledge  of  acquisition  regulations.  The  officers  with  no 
operational  experience  felt  the  same  way,  but  they  also  felt 
strongly  (a^.l)  that  knowledge  of  the  threat  was  more 
important  than  knowledge  of  acquisition  regulations .  The 
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same  test  indicated  that  this  last  group  of  officers  also 
felt  strongly  that  knowledge  of  technology  to  make  the 
weapon  systems  work  was  more  important  than  knowledge  of  the 
acquisition  process. 

Table  10 

Ranking  of  Knowledge  Areas  by  Samples 


No  Operational  Exp. 


nol 


2 

Tactics  of  Deployment 

2 

3 

Maintenance 

3 

Maintenance 

*-i 

Threat 

4 

Threat 

5 

Acquisition  Process 

5 

Acquisition  Process 

6 

Acquisition  Regs 

Acquisition  Regs 

vv  =  . 


Rated 


Threat 


Tactics  of  Deployment 


Acquisition  Process 


a  u  Cl  —  .  O 


Ma intenance 


Acquisition  Regs _ 

W  =.139,  not  stat  sig 


*  Tied  Ra.nkinc 


Maintenance 


Threa 


Acquisition  Process 


Tactics  of  Deployment 


Acquisition  Reqs 


W  -.172  not  stat  sig 


The  result  for  the  rated  officers  arid  other  ncn-rated 
officers  with  operational  experience  indicate  that  there  wu; 
very  little  agreement  in  the  way  they  ranked  the  six 
knowledge  areas.  None  of  the  statistical  tests  applied 
indicated  any  statistical  significance. 


Summary 


This  chapter  described  the  results  of  the  analysis  of 
the  data  collected  with  the  survey.  The  results  indicated 
that  officers  from  the  BEST  sample  ranked  themselves  higher 
than  officers  with  no  operational  experience  in  one  out  of 
the  five  questions  measuring  the  ability  to  interpret 
operational  requirements.  However,  rated  officers  ranked 
themselves  higher  than  officers  with  no  operational 
experience  in  two  out  of  these  five  questions. 

Officers  from  the  BEST  sample  ranked  themselves  better 
in  the  questions  measuring  the  ability  to  communicate  with 
the  using  organization.  In  this  area,  officers  from  the 
BEST  sample  ranked  themselves  higher  than  officers  with  no 
operational  experience  in  three  out  of  the  five  questions, 
while  rated  officers  ranked  themselves  higher  in  only  one 
question . 

On  the  questions  measuring  whether  operational 
experience  helps  S&DE  officers  establish  credibility  for 
their  judgement,  only  the  rated  officers  ranked  themselves 
higher  than  officers  with  no  operational  experience  in  cr.e 
question  . 

In  the  effort  to  determine  the  relevancy  of  cperaticna 
experience  to  specific  5i*DE  jobs,  the  intended  methodology 
was  not  successful.  Only  two  of  the  officers  with 
operational  experience  expressed  that  their  operational 
experience  was  not  relevant  to  their  current  jobs. 


The  ranking  schemes  included  in  the  survey  provided 
some  insight  regarding  the  importance  of  operational  skills 
and  knowledge  to  the  officers  surveyed.  In  ranking  the 
skills,  officers  from  the  BEST  sample  ranked  operational 
skills  number  two,  while  officers  with  no  operational 
experience  ranked  it  number  five.  The  results  from  the 
other  two  samples  were  inconclusive. 

In  ranking  the  knowledge  areas,  officers  in  both  the 
BEST  sample  and  officers  with  no  operational  experience 
ranked  the  knowledge  areas  in  the  same  way.  Officers  in 
both  groups  ranked  the  two  knowledge  areas  associated  with 
operational  knowledge  second  and  third.  The  results  from 
the  two  other  samples  indicated  lower  agreement  as  to  how 
officers  in  these  samples  perceived  the  importance  of  each 
of  the  knowledge  areas. 

Chapter  V,  Conclusions  and  Recommendations,  will 
present  the  final  outcome  of  this  research  study.  The 
conclusions  and  recommendations  presented  in  the  next 
chapter  are  based  on  the  outcome  of  the  analyses  conducted 
in  this  chapter. 


V. 


Conclusions  and  Recommendations 


Introduction 

This  research  study  was  undertaken  to  determine  if  the 
operational  experience  gained  by  S&DE  officers  through  the 
Broadening  Experience  Tour  (BEST)  program  enhances  their 
ability  to  develop  useful  operator-oriented  systems.  The 
officially  stated  benefits  of  operational  experience  to  S&DE 
officers  were  found  in  Air  Force  Regulation  36-23,  Off icer 
Professional  Development.  These  benefits  are  the  enhanced 
ability  to  interpret  operational  requirements,  communicate 
with  the  using  organizations,  and  to  help  establish 
credibility  for  their  judgement. 

The  results  of  this  study  do  not  apply  .to  officers  in 
all  ranks  in  S&DE  career  fields.  Inferences  made  apply  only 
to  officers  with  six  to  ll  years  of  active  duty  commissioned 
service  time.  This  chapter  presents  a  summary  of  key 
findings  followed  by  implications.  The  limitations  of  the 
study,  and  topics  not  addressed  by  this  study  effort,  are 
then  p  esented,  followed  by  the  conclusion  of  the  study. 
Finally,  recommendations  for  further  research  studies  are 
presented . 

Summary  of  Key  Findings 

A  summary  of  key  findings  is  presented  as  each  pertains 
to  a  specific  survey  question.  Inferences  cn  the 


possible  reasons  for  the  results  from  the  survey  questions 
are  presented  as  well. 

Finding  1.  Officers  from  the  BEST  sample  ranked 
themselves  higher  than  officers  in  all  other  samples  in 
their  ability  to  evaluate  logistics  support  requirements. 

All  the  officers  in  the  BEST  sample  had  experience  as 
aircraft  maintenance  officers,  which  should  explain  why  they 
ranked  higher  than  officers  in  all  other  samples.  Although 
the  rated  officers  had  more  years  of  experience  with  the 
actual  weapon  systems,  their  knowledge  of  logistics  support 
was  lower.  Although  it  is  not  known  if  aircraft  maintenance 
was  the  first  choice  of  operational  assignment  of  these 
officers,  logistics  support  is  an  area  that  is  always 
stressed  in  acquisition  training  and  education. 

Finding  2 .  Officers  in  the  rated  sample  were  the  only 
officers  to  rank  themselves  higher  than  officers  with  no 
operational  experience  in  their  ability  to  read  and 
understand  operational  cie  f  lciencies  outlined  in  operational 
requirements  documents.  Understanding  of  operational 
requirements  probably  entails  a  broad  knowledge  of  mission 
requirements.  It  is  possible  that  rated  experience  provides 
tetter  knowledge  for  the  understanding  of  mission 
requirements  than  ncn-rated  operational  experience. 

Finding _ 3.  There  was  no  difference  in  the  ability  cl 

officers  with  and  without  operational  experience  to  evaluate 
technical  solutions  to  operational  deficiencies.  However, 
officers  m  the  bf.U'i  sample  ranked  higher  than  o!  risers  with 


other  non-rated  operational  experience  in  this  area.  The 
difference  detected  between  these  two  samples  could  be 
caused  by  more  than  one  factor.  The  sample  of  officers  with 
other  non-rated  operational  experience  had  in  general  less 
acquisition  experience,  and  were  mostly  program  managers. 

The  officers  in  the  BEST  sample  had  more  acquisition 
experience  and  were  mostly  engineers. 

Finding  4.  No  difference  was  found  between  any  of  the 
samples  of  officers  surveyed  in  their  ability  to  assess  the 
appropriateness  of  performance  parameters  outlined  in  system 
level  specifications  and  other  technical  documents.  It  is 
possible  that  the  operational  knowledge  required  to  assess 
performance  parameters  could  be  acquired  without  being  a 
user  or  a  maintainer  ot  a  weapon  system.  The  officers  with 
no  operational  experience  had  in  general  more  acquisition 
experience  than  officers  with  operational  experience.  It  is 
possible  through  the  course  of  their  acquisition  careers 
that  these  officers  gained  the  knowledge  required  to  make 
appropriate  assessments  of  performance  parameters. 

F i nd 1 ng  5 .  Officers  in  the  rated  sample  were  the  only 
officers  to  rank  themselves  higher  than  officers  with  no 
operational  experience  m  their  ability  to  evaluate  test 
requirements  needed  to  assess  the  military  worthiness  ot 
weapon  systems/components.  On  the  other  hand,  supervisors 
of  officers  with  no  operational  experience  ranked  their 
subordinates  higher  than  the  supervisors  ot  the  officers  in 
the  lildlT  sample.  The  fact  that  the  rated  officers  ranked 


themselves  higher  than  officers  with  no  operational 
experience  may  indicate  that  the  type  of  operational 
experience  is  operative  in  enhancing  the  ability  to  evaluate 
test  requirements. 

Finding  6.  Officers  in  the  rated  sample  were  the  only 
officers  to  rank  higher  than  officers  with  no  operational 
experience  in  their  ability  to  understand  the  language  used 
in  the  operational  requirements  documents. 

Finding  7 .  Officers  from  the  BEST  sample  were  the  only 
officers  to  rank  themselves  higher  than  officers  with  no 
operational  experience  in  their  ability  to  understand  the 
acronyms  used  by  users  and  maintainers  of  weapon  systems. 

The  higher  response  from  the  BEST  sample  could  be  attributed 
to  the  homogeneous  composition  of  the  BEST  sample  (all 
maintenance  officers),  operational  exposure  (contact  'with 
users)  or  to  the  present  assignment  of  these  officers. 

Finding  8.  Officers  from  the  BEST  sample  were  the  only 
of: icers  to  rank  themselves  higher  than  officers  with  no 
operational  experience  m  their  ability  to  determine  the 
organization  in  the  using  command  which  can  provide  the 
various  types  of  data  required  to  do  their  work.  One 
possible  reason  for  this  result  is  the  type  or  operational 
experience.  Although  the  rated  of  risers  had  more  years  or 
operational  experience  than  the  officers  in  the  BEST  sample, 
it  is  possible  that  the  officers  in  the  BEST  sample  had  to 
interface  with  more  organizations  in  the  using  command  than 
rated  officers  in  order  to  do  their  jobs. 
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Finding  9.  Officers  from  the  BEST  sample  were  the  only 
officers  to  rank  higher  than  officers  with  no  operational 
experience  in  their  ability  to  determine  the  type  of  data 
required  by  maintenance  personnel  to  support  weapon  systems. 
Again  this  result  appears  to  be  caused  by  the  fact  that  all 
officers  in  the  BEST  sample  were  aircraft  maintenance 
of  f icers . 

Finding  10.  Officers  from  the  rated  sample  were  the 
only  officers  to  rank  higher  than  officers  with  no 
operational  experience  in  their  ability  to  influence 
decisions  affecting  the  development/production  of  weapon 
systems/components . 

A  possible  reason  why  rated  officers  and  not  officers 
from  the  BEST  sample  ranked  higher  in  their  ability  to 
influence  development/ product  ion  decisions  could  be 
attributed  to  the  composition  of  these  two  groups.  The  vast 
majority  of  rated  officers  were  program  managers  while  the 
majority  of  officers  in  the  BEST  sample  we  re  engineers.  In 
addition,  the  majority  of  the  officers  with  no  ope rat  ion a  1 
experience  were  engineers.  Major  decisions  concerning 
development  and  production  are  normally  made  by  program 
management  not  c-ng  ineer  i  ng  . 

Finding  11.  Although  there  was  a  difference  (at  a= .  OS) 
in  only  one  of  the  five  questions  designed  to  measure 
credibility,  the  results  revealed  a  consistent  pattern.  The 
responses  from  all  samples  were  close  when  responding  to 
questions  measuring  the  ability  to  provide  adv  ice  and 


opinions  on  matters  and  decisions  affecting  capabilities  and 
the  development/product  ion  of  weapon  systems.  However,  when 
asked  about  their  ability  to  influence  decisions,  officers 
with  operational  experience  rated  themselves  higher  than 
officers  without  it.  It  is  possible  that  operational 
experience  is  operative  in  the  perceived  ability  of  these 
officers  to  influence  decisions. 

Finding  12.  Despite  the  fact  that  survey  question  45 
could  not  be  used  in  the  analysis,  the  results  obtained  for 
investigative  questions  one  through  three  provide  sene 
inright  into  the  relevancy  of  specific  operational 
ex;erience.  The  non-rated  officers  who  obtained  their 
operational  experience  prior  to  entering  the  S&DF  career 
fields  consistently  ranked  themselves  lower  than  the 
other  officers  with  operational  experience  ir.  12  out  cf  the 
15  measurement  questions. 

The  sample  of  officers  with  other  non-rated  operational 
experience  was  the  most  heterogeneous  in  terms  of  specific 
operational  experience.  There  were  three  aircraft 
maintenance  officers,  three  missile  officers,  and  one 
intelligence  officer.  Ir.  addition,  the  two  officers  who 
indicated  that  their  operational  experience  was  i r re  levant 
also  belonged  to  this  sample.  It  could  be  argued  that 
perhaps  a  lack  of  acquisition  experience  might  have  boon  the 
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majority  of  the  officers  in  both  groups  had  the  same  S&DE 
AFSC  ( 2  7  X  X )  . 

Further,  all  rated  officers  had  experience  as  users  of 
aeronautical  systems  which  are  the  main  products  of  ASD.  In 
addition,  the  responses  of  all  three  groups  of  officers  with 
operat  al  experience  were  combined  and  compared  'with  the 
resoonse  ;  from  the  officers  'with  no  operational  experience. 
The  median  tests  conducted  did  not  indicate  any  significant 
statistical  differences  between  the  median  scores  of 
officers  with,  operational  experience  (when  combined;  and 
those  without  it.  This  Last  finding  suggests  that  relevancy 
of  ■  aecific  experience  could  be  an  influential  factor. 

The  way  in  which  the  measurement  questions  were 
constructed,  could  also  provide  seme  msignt  into  the 
relevance  of  specific  operational  experience.  Quest  ion  11 
turned  out  tc  b<  more  specific  then  intended.  Question  1] 
i, .ensured  the  ability  tc  evaluate  logistics  support.  The 
BEST  o£  i  ice.,  s,  ail  of  which  had  aircraft  maintenance 
experience,  ranked  themselves  higher  than  the  oil icers  .n 
all  other  samples.  This  imding  suggests  tint  n  the  ..  ob 
requires  sdoc  i  l  ic  knowledge  o'  icq;.. tic  support. ,  an 
with  aircraft  mu  :  nr '-I.  i  •  •  j  i  cl:  I  i  ■  .  -:::p  f  •  i  . 

Therefore ,  those  otiK-  r  uii  inn  •  i  •  -j  r<-.  aits  tm-i  to  cup  pi;  it. 
the  promise  that;  ;  p*  •«.:  :  i  o  j  <  •  i  ■  i  *"  • «  r.  i!  •<  •:  i  enc<_  i  c>  r-_-  i  evanf 
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operational  experience.  It  would  be  difficult  to  assess  to 
what  extent  this  result  is  attributed  to  the  experience 
gained  through  the  BEST  program;  it  could  be  assumed  that 
operational  experience  was  important  to  these  officers  since 
they  volunteered  for  the  program. 


Impl  icat ions 

This  section  presents  possible  implications  derived 
from  the  key  findings  presented  in  the  preceding  section. 

Due  to  the  research  design  applied  in  this  study,  the 
implications  presented  here  are  applicable  only  to  officers 
with  six  to  11  years  ol  active  duty  commissioned  time.  The 
specific  ir.pl  icat  ions  are  presented  below: 

1.  Since  aircraft,  maintenance  experience  appears  to  have 
the  rest  ir.pact  in  enhancing  communication  abilities, 
officers  with  28XX  primary  AFSCs  returning  from  aircraft 
maintenance  duties  should  be  allowed  to  crossflow  to  the 
2 7 XX  career  field  early.  The  rationale  for  this  implication 
is  that  officers  in  27XX  career  fields  have  to  communicate 
with  a  wider  variety  of  organizations  than  officers  in  2 3 XX 
career  fields.  Therefore,  the  officer's  operational 
experience  can  be  used  where  it  will  be  most  beneficial. 

: .  Rated  officers  should  be  assigned  to  jobs  that  require 
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7.  Operational  experience  in  general  does  not  enhance  the 
credibility  of  S&DE  officers  for  their  judgment. 

8.  Operational  experience  is  best  utilized  when  relevant  to 
the  S&DE  assignment. 

Lim itat ions 

This  research  study  focused  on  areas  of  acquisition 
where  operational  experience  is  believed  to  help  the  most. 
Therefore,  many  skills  required  in  acquisition  were  not 
studied.  Seme  of  the  skills  net  included  in  this  study  are: 
the  ability  to  schedule  o.r.d  monitor  contract  actions;  the 
ability  to  manage  financial  resources;  the  ability  develop 
acquisition  strategies;  and  the  ability  to  conduct 
successful  design  reviews.  Therefore,  further  research  is 
required  to  fully  measure  the  impact  of  operational 
experience . 


Conclusion  of  Stud' 


The  specific  implications  presented  in  this  chapter 
serve  as  the  foundation  for  the  general  conclusion  that  the 
operational  experience  obtained  by  SIDE  officers  through  the 
BEST  program  has  some  effect  in  enhancing  their  ability  to 


develop  useful  operator-oriented  systems 


ao:  me 


Force  Regulation  36-2  j .  Further,  their  ubiiitii 


a  I  G  IdG  G  t 


..nnonced  with  respect  to  communicating  with  the  using 
commands.  The  long  term  el  loots  of  operational  oxp-or  i  o  two 
acquired  through  the  BEET  program,  on  the  various  abilities 
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or.lv  to  the  r- 

e  r  i  j 
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-  i  i  i  1  ’ 

immediately  following  the  return  of  these  officers  from 
operational  jobs.  It  is  possible  that  being  absent  from 
acquisition  for  four  years  could  have  an  effect  in  their 
ability  to  immediately  realize  the  full  potential  of  their 
operational  experience. 

It  is  possible  that  operational  experience  has  a 
greater  impact  in  the  abilities  of  higher  ranking  S&DE 
officers,  but  this  is  a  subject  for  future  study. 

Recommendation  for  Further  Research 

The  main  recommendation  based  on  the  results  of  this 
research  effort  is  to  conduct  a  longitudinal  study  of  the 
effects  of  operational  experience  on  the  abilities  of  SwDE 
officers.  However,  for  the  recommended  action  to  be 
successful  there  must  be  complete  support  and  dedication  to 
the  study  by  the  personnel  offices  at  the  Air  Force  Materiel 
Command  and  Air  Force  Headquarters,  since  such  a  study  .  ill 
require  years  to  complete.  A  survey  should  be  admin i stored 
to  the  officers  selected  to  participate  in  the  DF ST  program 
prior  to  the  start  of  their  operational  assignment.  Ihe 
same  survey  should  then  be  administered  some  time  after 
returning  to  ;J£DR  duties,  an:  the  results  compared  to 
measure  for  treatment  ci loots. 

In  addition,  the  survey  snould  a  Luo  be  administered  to 
ail  D&UJi  olticers  with  operational  experience  t.o  determine 
the  point  in  their  career  whore  oooraticnal  experience  is 
most  use  till  ;  and  to  determine  tin*  et loots  which  the 


relevancy  of  that  specific  operational  experience  has  on  the 
abilities  of  S&DE  officers.  The  effects  that  specific 
operational  experience  might  have  in  the  effectiveness  of 
S&DE  officers  in  the  different  product  divisions  should  be 
studied.  This  research  is  necessary,  because  of  the 
reductions  in  force  expected  in  the  near  future.  The  full 
effects  of  operational  experience  should  be  studied  and 
understood  so  the  Air  Force  can  wisely  pursue  the  infusion 
of  the  operational  point  of  view  at  a  nininun  cost  to  the 
Air  Force. 

The  suggested  research  study  should  start  with  an 
effort  to  identify  any  other  possible  benefits  of 
operational  experience.  Tt.  is  not  clear  how  the  benefits 
outlined  in  AFR  36-23  were  determined  or  how  encompassing 
they  are.  The  delphi  technique  should  be  applied  by  a  group 
of  S&DE  officers  and  supervisors  at  various  levels  to 
determine  all  the  perceived  benefits  of  operational 
experience.  Once  these  benefits  are  identified,  another 
session  should  be  assembled  to  determine  how  to  construct 
the  measurement  questions. 

Cnee  the  new  survey  is  constructed  and  pretested,  it 
should  be  sent  to  officers  of  all  ranks  both  with  and 
without  operational  experience  in  all  product,  divisions  cl 
Air  Force  Systems  Command.  The  results  should  then  bo  used 
to  study  the  et  tects  of  operational  experience  at  various 
points  in  the  careers  ot  S&DL  officers. 

/  • ) 


Anpenriix  A.  Subordinate  Survey 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AERONAUTICAL  SYSTEMS  DIVISION  <AF5C) 
WRIGHT  PATTERSON  AIR  FORCE  BASE.  OHIO  45-133-6503 


SuLvey  on  Job  Skills 


O L  nCCU 


i  t  ion  Personnel 


iu*J£C' 


July  1)51 


Survey  Recioient 


1.  On  26  December  1990 .  AF  Regulation  >>-27.  Acquisition  Professional 
De-.  elopmen  t .  'was  published  in  recognition  ot  the  need  to  improve  the 
preparation  of  all  personnel  involved  in  the  acquisition  of  veanon  SVsr< 

survey ,  you  are  in  a  position 
:o  a id  in  the  process  of 
professional  development  of  acquisition  pe r tonne i  .  The  data  collected 
will  be  used  to  identity  the  critical  factors  attesting  tn.e  ability  of 
acquisition  personnel  tc  develop  effective  weapons  sys ferns  for  tne’  Ait- 


in  the  Air  Force.  As  a  recipient  of  tin: 
contribute  to  a  research  project  set  out 


?c 


1  S 


to 


w .  Please  tane  lii  -  if  minutes  to  ecsolet^  the  attached  s 1 1 1  v  •“ '  -md  r^i-ir"- 
it  Ji_T  l.t  July  199i.  four  individual  response  will  be  treated  as 
con: iden t ial . 

o  •  i o 1  u l  par  t  .c .  on  <.  :*;n  in  <.  a  i  s  s  . u ti  v  .  s  c 1  > rn 1. 1  e  t  e  r  v  v o . u n  t  a i  ’ * ,  b  i  *  v e  would 
greatly  appreciate  your  help.  For  former  inf'U'tin  t  i  >n ,  nlease  -con  to--  th° 
faculty  adviser  for  mis  resea ren  project.  C.uit  Kevin  •.> ;k: r  at  Dili!  j-33' 


Program  Manage non t 
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SURVEY  1  July  1991 

ON 

JOB  SKILLS 

OF  ACQUISITION  PERSONNEL 
Instructions 

Please  answer  all  questions.  Read  each  question  carefully 
before  selecting  your  answer.  Use  a  number  two  pencil  to 
circle  your  answer  on  the  survey  and  to  darken  the  circle  on 
the  machine  scoreable  answer  sheet  (AFIT  Data  Collection 
Form).  As  you  complete  the  survey,  make  sure  your  answers 
on  the  answer  sheet  match  the  questions  you  are  actually 
answering.  If  you  decide  to  change  any  of  your  answers, 
erase  the  incorrect  mark  completely  on  both  the  survey  and 
the  answer  sheet.  Please  do  not  fold,  staple,  or  tear  the 
answer  sheet. 

To  ensure  your  response  remains  anonymous,  please  do  not 
write  your  name  or  social  security  number  on  either  the 
survey  or  the  answer  sheet. 

Please  do  not  discuss  the  results  of  this  survey  with  ycur 
peers  or  supervisor. 

Background  Information 


1.  What  is  your  current  rank? 

a.  First  Lieutenant 

b.  Captain 

c.  Major 

d.  Lieutenant  Colonel 

e.  Colonel 

2.  What  is  your  current  duty  AFSC? 


a  . 

2  6XX 

d.  65XX 

b . 

2  7  XX 

e .  67  X  X 

c . 

2  3  XX 

Hew 

many  years  ol  acquisition  experier,' 

ce  (experience  in 

’ ,  2  7  X  X ,  2  8  X  X  , 

6  5  X  X  ,  & 

6  7  XX 

utilization 

fields)  do  you 

less 

-> 

a  . 

Less  than 

2  years 

b . 

2  years, 

but  less 

than 

4  years 

c . 

4  years, 

but  less 

than 

6  years 

d. 

6  years. 

but  less 

than 

6  years 

e  . 

8  years. 

but  less 

t  h  a  n 

10  years 

10  years 

or  more 

/  O 


4.  Indicate  your  aeronautical  rating? 

a.  Pilot 

b.  Navigator 

c.  None 

5.  Indicate  if  you  possess  a  primary,  secondary,  or 
tertiary  AFSC  in  any  of  the  following  utilization  fields: 

(If  you  possess  more  than  one  of  the  following  AFSCs,  please 
indicate  the  one  where  you  spent  the  most  time.) 


a . 

1 6XX 

f  . 

3  1XX 

j .  none 

b. 

1 7  XX 

g  • 

4  OXX 

c . 

18XX 

h . 

3  OXX 

d. 

e . 

2  OXX 

2  5XX 

i . 

8 1XX 

6.  How  many  years  of  operational  experience  do  you  have? 
(Operational  experience  is  defined  as  years  performing  th 
duties  of  pilot,  navigator  or  the  duties  in  any  of  the  AF 
listed  in  question  5) 


a . 

Less  than 

2 

years 

b. 

2  years, 

but 

less 

than 

4 

years 

c . 

4  years. 

but 

less 

than 

6 

years 

d . 

6  years, 

but 

less 

than 

a 

years 

e . 

8  years, 

but 

less 

than 

10 

years 

f . 

10  years 

or 

more 

g- 

none 

7.  To  which  of  the  following  organizations  are  you 
currently  assiqned? 

•  a.  AD  d.  ESD 

b.  ASD  e.  MSD 

c.  BMO  t.  Other 

o.  What  is  your  primary  undergraduate  education? 

a.  Technical  (i.e.  engineering,  science-related) 

b.  Non-technical  (i.e.,  humanities,  or  business 
related) 

c.  Both  technical  and  non-technical  \i.e. ,  two 
different  degrees) 

9.  What  is  your  primary  graduate  education? 

a.  none 

b.  Technical  (i.e.,  engineering,  science- related) 

c.  Non-technical  (i.e.,  humanities,  or  business- 
related) 

d.  Beth  technical  and  nen-teehn  lea  i  <  i  .  e  .  two  Jit;  orer.t 
deg  roes ) 


Study  Questions 


EXAMPLE 


The  following  is  an  example  of  the  correct  way  to 
answer  questions  10  to  32: 


85.  When 
relations 
than 


it  comes  to 
with  others 
of  my  peers 


having  good  interpersonal 
in  my  organization,  I  do  better 
whose  jobs  are  similar  to  mine. 


<40%  50%  60%  70%  80%  90%  95% 

I  I _ ! _ I _ I _ I _ I 

I  I  I  I  I  '  I 

a.  b.  c.  d.  e.  f .  g . 


Selection  of  response  e.  would  indicate  that  you  do 
better  than  80  percent  of  your  peers  whose  jobs  are  |j 
similar  to  yours  when  it  comes  to  having  gccd 
interpersonal  relations  with  others  in  your  ^ 

organization.  i| 


10.  When  it  comes  to  providing  advice  on  matters  affecting 
the  development/production  of  the  weapon 

systems ( s )/ component ( s )  in  my  organization,  I  am  called  upon 


more 

mine , 

often 

than 

of 

my 

peers  whose  jobs  are  similar  to 

<40% 

50% 

CTv 

O 

70% 

80 

%  90%  95% 

1  ! 

a . 

1 

b. 

l 

C  . 

1 

d  . 

e  . 

f .  g. 

1  1 

1;  her. 

it  comes  to  o' 

/a  1 

inn  ti  o  i_  c  g  i.  c  t  i  o  s  o  u  c  c  c  r  t 

required  to  maintain  the  weapon  system  (S;  /  component s,  that 

xy  organization  develops/produces,  I  do  better  than  _  of 

my  peers  whose  jobs  are  similar  to  mine. 


<4  0\ 

i 

50% 

ovO 

o 

o  - 

7  0  % 

8  0% 

90% 

9  5  l 

i 

i 

a . 

1 

b  . 

e . 

d  . 

e . 

i  . 

i 

g  • 

12.  When  it  conies  to  influencing  decisions  affecting  the 
capabilities  of  the  weapon  systems (s) /component (s)  that  my 

organization  develops/produces,  I  do  better  than  _  of  my 

peers  whose  jobs  are  similar  to  mine. 


<4  0% 

i 

50% 

i 

60% 

_ t _ 

70% 

i _ 

80% 

i 

90% 

_ i _ 

95% 

_  i 

i 

a  . 

t 

b. 

i 

c . 

i 

d. 

i 

e . 

i 

f  . 

i 

g- 

13  . 

When 

it  comes  to 

briefing  the 

users  of  the  weapon 

system(s)/  component(s)  that  my  organization 

develops/produces,  I  do  better  than  _  of  mv  peers  whose 

jobs  are  similar  to  mine. 


<40% 

50% 

i 

60% 

i 

7  0  % 

oY» 

O  - 
CO 

kO 

-•  o 
o\o 

95% 

1 

1 

a  • 

i 

b. 

i 

C  . 

1 

d  . 

e . 

1 

* 

1 

g. 

14.  When  it  comes  to  reading  and  comprehending  the 
operational  deficiencies  outlined  in  operational 
requirements  documents  such  as  the  Statement  of  t.'eed  (SOU;  , 
the  Statement  of  Operational  Requirements  Document  (SORD) , 
Operational  Requirements  Document  (ORD) ,  and  the  Mission 


Need 

Statement 

/  M  M  Q  \ 

\  *  “ 1  /  t 

I  do 

be  1 1  e  it 

th  a  r. 

whose 

j  obs 

are 

similar 

to 

mine . 

<40% 

1 

50% 

60% 

7  0  % 

8  0% 

90% 

9  5% 

i 

1 

a . 

1 

b. 

l 

C  • 

1 

d. 

i 

e . 

1 

f  . 

i 

g- 

15.  When  it  comes  to  communicating  clearly  in  my 

organization,  I  do  better  than  _  of  my  peers  whose  jcbs 

are  similar  to  mine. 


-  IA 

Xw 

O 

50% 

6  0  %  7  0  % 

9  0% 

1 

95% 

i 

a  . 

1 

b. 

1  :  1 
c  .  d  .  e  . 

1 

f  . 

g- 

16  . 

When 

it  ccmes  to  ho); 

mg  cthe 

rs  to  3 ; 

a *ve  problems 

orga 

nizat 

ion,  I  do  better 

than 

of  my 

poors  w hose  j o  b 

are 

s  in  i  1 

a r  to  mine. 

<40% 

5  0% 

i 

60% 

7  0% 

o  0  % 

9  0% 

9  5% 

a  . 

b. 

c . 

d. 

i 

G  . 

t  . 

i 

g- 

r»  i 
O  1 


17 .  When  it  comes  to  understanding  the  language  used  in 
operational  requirements  documents  such  as  the  Statement  of 
Need  (SON) ,  the  Statement  of  Operational  Requirements 
Document  (SORD) ,  Operational  Requirements  Document  (ORD) , 

and  the  Mission  Need  Statement  (MNS),  I  do  better  than  _ 

of  my  peers  whose  jobs  are  similar  to  mine. 


<40% 

i 

50% 

i 

60% 

i 

70%  30% 

i  i 

9  0% 

95% 

_  t 

i 

a . 

i 

b. 

i 

c. 

i  i 

d .  e . 

f . 

i 

g* 

18  . 

When 

I  attend  program 

reviews 

related  to  the  weapon 

system ( s) /component ( s ) 

that 

my  orga 

nization 

develops/ 

produces,  I  contribute 
are  similar  to  mine. 

more 

than  _ 

_  of  my 

peers  whose  jobs 

<40%  50%  60%  70% 

!  1  1 

80% 

i 

90% 

9  5% 

_  i 

1  t  !  1 

a  .  b  .  c .  d  « 

e . 

1 

f  . 

q  • 

19.  When  evaluating  alternative  technical  solutions  to  the 
operational  deficiencies  which  will  be  corrected  by  the 
weapon  system ( s )/ component ( s)  that  my  organization 

develops/produces,  I  do  better  than  _ _  of  my  peers  whose 

jobs  are  similar  to  mine. 


<4  0% 

i  _ 

50% 

i 

6  0% 

70% 

30% 

90% 

i 

a  . 

i 

b. 

c . 

d. 

e  . 

1 

f  . 

95' 

.  _  i 


i 

g 


20.  When  it  tomes  to  providing  advice  cn  matters  affecting 
the  capabilities  of  the  weapon  system ( s) /component ( s)  that 

illed  upon  more 


my  org 

a  n  i  z  a  t 

ion  dev 

elops/ produces, 

I  am 

o  f  ten 

than 

_ of 

my 

peers 

whose 

j  obs 

<4  0% 
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5  0%  6 
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90% 
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95% 
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i  i 
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Air 

Fore 
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Fed 
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Acquisi 

t  ion 

I  know 

more 

than 

of  my 
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mine. 
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o 
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22.  When  assessing  the  appropriateness  of  performance 
parameters  outlined  in  system  level  specifications  and  other 

technical  documents,  I  do  better  than  _  of  my  peers  whose 

jobs  are  similar  to  mine. 


<40% 

50% 

..  i 

60% 

i 

70% 

i 

80% 

i 

90% 

-  i  _ . 

95% 

! 

1 

a . 

i 

b. 

i 

c . 

i 

d. 

i 

e , 

i 

f . 

i 

g- 

23.  When  it  comes  to  overall  contribution  to  the  mission  of 

my  organization,  I  contribute  more  than  _  of  my  peers 

whose  jobs  are  similar  to  mine. 


-  IA 

1  J- 
1  O 

I  o\° 

II 

50% 

_  i _ 

60% 

_ i _ 

70% 

__  i  _ 

8  0% 

1 

90% 

95% 

_ i 

1 

a . 

b. 

i 

c . 

i 

d . 

.  | 
e . 

f . 

i 

g- 

24._ When  it  comes  to  understanding  the  acronyms  used  by  the 
users  and  maintainers  of  the  weapon  system ( s) /component ( s ) 
that  my  organization  develops/produces,  I  do  better  than 
_  of  my 


peers 

whose 

j  obs 

are 

similar 

to  mine. 

<40% 

50% 

_  i 

60% 

i  -  _ . 

7  0% 

-  i  — - 

80% 

i 

90% 

95% 

i 

a . 

i 

b. 

i 

C  - 

i 

d. 

e . 

i 

f . 

g  • 

25.  When  assessing  technical  risks  involved  in  the 
development/production  of  weapon  system ( s ) /component ( s )  in 

my  organization,  I  do  better  than  _  of  my  peers  whose  jobs 

are  similar  to  nine. 
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26.  When  evaluating  the  test  requirements  needed 
to  assess  the  military  worthiness  ot  the  weapon  system  v  s) 
component (s)  that  my  organization  develops/ produces,  I  do 
better  than  _  of  my  peers  whose  jobs  .are  similar  to  mine. 
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27.  When  it  comes  to  determining  the  organization  in  the 
using  command  which  can  provide  the  various  types  of  data 
required  to  do  my  work,  I  do  better  than  _  of  my  peers 


whose 

jobs 

are 

similar 

to  mine. 
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28  . 

When 

it  comes  to 

encouraging 

open 

the  team  members  in  my  organization,  I  do  better  than 


of  my 

peers 

whose 

jobs  are 

similar  to 
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29  . 

When  it  comes  to  knowledge  of  the 

required  to 

make 

the  weapon 

system ( s) / 

organization  develops/ produces  work.,  I 

o  f  my 

peers 

whose  jobs 

are  sir. i la 
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3 C .  When  it  comes  to  providing  my  opinion  on  decisions 
affecting  the  capabilities  of  the  wezpon 
system ( s) /component ( s)  that  my  organization 

develops/prcduces ,  I  am  called  upon  more  often  than  _ 

my  peers  whose  jobs  are  similar  to  mine. 
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32.  When  it  comes  to  influencing  decisions  affecting  the 
development/production  of  the  weapon  system (s) /component ( s ) 

in  my  organization,  I  do  better  than  _  of  my  peers  whose 

jobs  are  similar  to  mine. 

<40%  50%  60%  70%  80%  90%  95% 

I _ I _ _l _ ! _ I _ I _ I 

I  I  I  I  I  !  I 

a.  b.  c.  d.  e.  f.  g. 


33.-38.  How  important  do  you  think  the  following  skills  are 
to  officers  involved  in  the  development  and  production  of 
useful  operator-oriented  weapon  systems? 

Rank  the  following  skills  based  on  their  relative 
importance.  Record  the  numerical  rank  in  the  blank.  Assign 
a  different  number  to  each  of  the  six  skills.  Please  record 
the  letter  corresponding  to  each  rank  on  the  answer  sheet. 


Most 

Important 


Least 

Important 


a . 


c . 


d. 


e . 


1  D 


3  6 


Technical  skills 
Communication  skills 
Decision  making  skills 

Operational  skills  (  experience  in  the  us< 
maintenance  or  weapon  systems/ccmpononts; 

Administrative  skills 

Leadersh id  skills 


(Continue  on  next  page) 


39--  44.  How  important  do  you  think  the  following  knowledge 
areas  are  to  officers  involved  in  the  development  and 
production  of  useful  operator-oriented  weapon  systems? 

Rank  the  following  knowledge  areas  based  on  their  relative 
importance.  Record  the  numerical  rank  in  the  blank.  Assign 
a  different  number  to  each  of  the  knowledge  areas.  Please 
record  the  letter  corresponding  to  each  rank  on  the  answer 
sheet . 


Most 

Important 

1  2 


Least 
Important 
5  6 


_  Knowledge  of  the  technology  required  to  make  the 

weapon  system  work. 

_  Knowledge  of  the  maintenance  required  to  keep  the 

weapon  system  working. 

_  Knowledge  of  Federal  Acquisition  Regulations 

(FARs)and  the  Air  Force  regulations  relevant  to 
the  acquisition  of  weapon  systems. 

_  Knowledge  of  the  threat  affecting  the  capability 

the  weapon  system. 

_  Knowledge  of  the  acquisition  process. 

_  Knowledge  of  the  tactics  of  deployment  of  the 

weapon  system. 


45.  If  you  have  rated  or  non-rated  operational  experience, 
how  relevant  do  you  think  this  experience  is  to  the 
development  of  the  weapon  systems  that  your  organization 
develops? 

a.  Completely  Irrelevant 

b.  Not  so  Relevant 

c.  Relevant 

d.  Highly  Relevant 

e.  This  question  is  NOT  APPLICABLE  (i.e.,  no 
operational  experience). 

PLEASE  RETURN  THE  SURVEY  AND  ANSWER  SHEET  IN  THE  RETURN 
ENVELOPE  PROVIDED  THROUGH  LOCAL  DISTRIBUTION. 


THANKS  FOR  YOUR  HELP  AND  HAVE  A  NICE  DAY! 


Supervisor  Survey 


A n pcnJix  B. 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AERONAUTICAL  SYSTEMS  OIVISION  (AFSCI 
WRIGHT. PATTERSON  AIR  FORCE  BASE.  OHIO  4S433-6S03 


A«H1  TO 

<n»  or  CY 

sua-cc t  Survey  o.:  Job  Skills  of  Acquisition  Personnel 


1  July  1991 


The  Supervisor  Of: 


1.  On  26  December  1990,  AF  Regulation  36-27,  Acquisition  Professional 
Development,  was  published  in  recognition  of  the  need  to  improve  rhe 
preparation  of  all  personnel  involved  in  the  acquisition  of  weapon  systems 
in  the  Air  Force.  As  a  recipient  of  this  survey,  you  are  in  a  position  to 
contribute  to  a  research  project  set  out  to  aid  in  the  process  of 
professional  development  of  acquisition  personnel.  Vhat  is  needed  from 
you  is  to  rate  your  subordinate  in  the  categories  outlined  in  the  survey. 
The  data  collected  will  be  used  to  identify  the  critical  factors  affecting 
the  ability  of  acquisition  personnel  to  develop  effective  weapons  systems 
for  the  Air  Force.  This  is  an  AFIT  thesis  project  sponsored  by  HQ 
AFSC/DPR0A2. 

2.  Please  take  10  -  15  minutes  to  complete  the  attached  survey  and  return 
it  NLT  15  July  1991.  Your  individual  response  will  be  treated  as 

con  f iden  t ial . 

3.  Your  participation  in  this  study  is  completely  voluntary,  but  ve  would 
greatly  appreciate  your  help.  For  further  information,  please  contact  the 
faculty '  a^^^rfor  this  research  project,  Capt  Kevin  Giant  at  DSN  78  5  -  3  3 7 

GEORGE  R.  VINTER5  II,  Col,  USAF 
Deputy  Chief  of  Staff, 

Program  Management 


3  Atch 

1.  Survey 

2.  Answer  Siieet 

3.  Return  Envelope 


SURVEY  1  July  1991 

ON 

JOB  SKILLS 

OF  ACQUISITION  PERSONNEL 


Instructions 


Please  answer  all  questions.  Read  each  question  carefully 
before  selecting  your  answer.  Use  a  number  two  pencil  to 
circle  your  answer  on  the  survey  and  to  darken  the  circle  on 
the  machine  scoreable  answer  sheet  (AFIT  Data  Collection 
Form) .  A  you  complete  the  survey,  make  sure  your  answers  on 
the  answer  sheet  match  the  questions  you  are  actually 
answering.  If  you  decide  to  change  any  of  your  answers, 
erase  the  incorrect  mark  completely  on  both  the  survey  and 
the  answer  sheet.  Please  do  not  fold,  staple,  or  tear  the 
answer  sheet. 

To  ensure  your  response  remains  anonymous,  please  do  net 
write  your  name  or  social  security  number  on  either  the 
survey  or  answer  sheet. 


EXAMPLE 


The  following  is  an  example  of  the  correct  way  to 
answer  questions  1C  to  32: 

85.  When  it  comes  to  having  good  interpersonal 
relations  with  otners  in  my  organization,  my 

subordinate  does  better  than  _  of  his/her  peers 

whose  jobs  are  similar  to  his/hers. 
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a.  b.  c.  d.  e.  f.  g. 


Selection  of  response  e.  would  indicate  that  your 
subordinate  does  better  than  8C%  of  his/her  peers 
whose  jobs  are  similar  to  his/hers  when  it  comes  to 
having  good  int.erpersona  1  relations  with  others  in 
your  organization. 
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1.  When  it  comes  to  providing  advice  on  matters  affecting 
the  development/production  of  the  weapon  systems (s)/ 
component (s)  in  my  organization,  I  call  upon  my  subordinate 

more  often  than  _  of  his/her  peers  whose  jobs  are  similar 

to  his/hers. 
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2. 

When  it 

comes 

to 

evaluating  the 

logistics  support 

required  to  maintain  the  weapon  system ( s) /component (s)  that 
my  organization  develops/produces,  my  subordinate  does 

better  than  _  of  his/her  peers  whose  jobs  are  similar  to 

his/hers . 
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3.  When  it  comes  to  influencing  decisions  affecting  the 
capabilities  of  the  weapon  systems ( s) /component ( s )  that  my 
organization  develops/produces,  my  subordinate  does  better 

than  _  of  his/her  peers  whose  jobs  are  similar  to 

his/hers . 
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4.  When  it  comes  to  briefing  the  users  of  the  weapon 
systen(s)/  component(s)  that  my  organization  develops/ 

produces,  my  subordinate  does  better  tnan  _  of  his/her 

peers  whose  jobs  are  similar  to  his/hers. 
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5.  When  it  comes  to  reading  and  comprehending  the 
operational  deficiencies  outlined  in  operational 
requirements  documents  such  as  the  Statement  of  Need  (SON, 
the  Statement  of  Operational  Requirements  Document  (SORD), 
Operational  Requirements  Document  (ORD) ,  and  the  Mission 

i.'eed  Statement  (MNS),  my  subordinate  does  better  than _ 

of  his/her  peers  whose  jobs  are  similar  to  his/hers. 
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6.  When  it  comes  to  communicating  clearly  in  my 

organization,  my  subordinate  does  better  than  _  of 

his/her  peers  whose  jobs  are  similar  to  his/hers. 
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7.  when  it  comes  to  helping  others  to  solve  problems  in  my 

organization,  my  subordinate  does  better  than  _ _  of 

his/her  peers  whose  jobs  are  similar  to  his/hers. 
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8  . 

When  it 

comes  to 

understanding 

the  language  used  in 

operational  requirements  documents  such  as  the  Statement  of 
Need  (SON) ,  the  Statement  of  Operational  Requirements 
Document  (SORD),  Operational  Requirements  Document  (ORD) , 
and  the  Mission  Need  Statement  (MNS) ,  my  subordinate  does 

better  than  _  of  his/her  peers  whose  jobs  are  similar  to 

his/hers . 
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9.  when  my  subordinate  attends  program  reviews  related  to 
the  weapon  system ( s ) /component ( ?)  that  my  organization 

develops/produces,  he/she  contributes  more  than  _  of 

his/her  peers  whose  jobs  are  similar  to  his/hers. 
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10.  When  evaluating  alternative  technical  solutions  to  the 
operational  deficiencies  which  will  be  corrected  by  the 
weapon  system ( s ) /component ( s )  that  my  organization 

develops/produces,  my  subordinate  does  better  than  _  c: 

his/her  peers  whose  jobs  arc  similar  to  his/hers. 
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11.  When  it  comes  to  providing  advice  on  matters  affecting 
the  capabilities  of  the  weapon  system (s ) /component ( s)  that 
my  organization  develops/produces,  I  call  upon  my 

subordinate  more  often  than  _  of  his/her  peers  whose  jobs 

are  similar  to  his/hers. 
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12.  When  it  comes  to  knowledge  of  the  various  acquisition 
regulations  (i.e..  Air  Force  regulations.  Military 
Standards,  and  the  Federal  Acquisition  Regulations  (FARs)), 

my  subordinate  does  know  more  than  _  of  his/her  peers 

whose  jobs  are  similar  to  his/hers. 
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13.  When  assessing  the  appropriateness  of  performance 
parameters  outlined  in  system  level  specifications  and  other 

technical  documents,  my  subordinate  does  better  than  _  of 

his/her  peers  whose  jobs  are  similar  to  his/hers. 
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14.  When  it  comes  to  overall  contribution  to  the  mission  of 

my  organization,  my  subordinate  contributes  more  than  _ 

of  his/her  peers  whose  jobs  are  similar  to  his/hers. 
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15.  When  it  comes  to  understanding  the  acronyms  used  by  the 
users  and  maintainers  of  the  weapon  system ( s) /component ( s } 
that  my  organization  develops/produces ,  My  subordinate  does 

better  than  _  of  his/her  peers  whose  j obs  are  similar  to 

his/hers. 
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16.  When  assessing  technical  risks  involved  in  the 
development/production  of  weapon  system ( s ) /component ( s )  in 

my  organization,  my  subordinate  does  better  than  _  of 

his/her  peers  whose  jobs  are  similar  to  his/hers. 
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17.  When  evaluating  the  test  requirements  needed 

to  assess  the  military  worthiness  of  the  weapon  system(s)/ 
component(s)  that  my  organization  develops/produccs ,  my 

subordinate  does  better  than  _  of  his/her  peers  whose 

jobs  are  similar  to  his/hers. 
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18.  When  it  comes  to  determining  the  organization  in  the 
using  command  which  can  provide  the  various  types  of  data 
required  to  do  his/her  work,  my  subordinate  does  better  than 
_  of  his/her  peers  whose  jobs  are  similar  to  his/hers. 
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19  . 

When 

it  comes  to 

encouraging 

open  communication  between 

the  team  members  in  my  organization,  my  subordinate  does 

better  than  _  of  his/her  peers  whose  jobs  are  similar  to 

his/hers . 
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20.  W’hen  it  comes  to  knowledge  of  the  various  technologies 
required  to  make  the  weapon  system ( s) /component ( s)  that  my 
organization  develops/prod-'ces  work,  my  subordinate  does 

know  more  than  _  of  his/her  peers  whose  jobs  are  similar 

to  his/hers. 
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21.  When  it  :omes  to  providing  his/hers  opinion  on 
decisions  affecting  the  capabilities  of  the  weapon 
system ( s) /component (s)  that  my  organization 
develops/produces,  I  call  upon  my  subordinate  more  often 

than  _  of  his/her  peers  whose  jobs  are  similar  to 

his/hers . 
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22  . 

When 

determining 

the  type  of 

data 

maintenance  personnel  to  support  the  weapon 
system (s) /component (s)  that  my  organization 
develops/produces,  my  subordinate  does  better  than 
his/her  peers  whose  jobs  are  similar  to  his/hers. 
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23.  When  it  comes  to  influencing  decisions  affecting  the 
development/production  of  the  weapon  system ( s) /component ( s) 

in  my  organization,  my  subordinate  does  better  than  _  of 

his/her  peers  whose  jobs  are  similar  to  his/hers. 
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PLEASE  RETURN  THE  SURVEY  AND  THE  ANSWER  SHEET  IN  THE  RETURN 
ENVELOPE  PROVIDED  THROUGH  LOCAL  DISTRIBUTION. 

THANKS  FOR  YOUR  HELP  AND  HAVE  A  NICE  DAY! 


Appendix  C:  Output  of  Cronbach's  Alpha  Computations 


Computations  for  Complete  Measuring  Scale 

The  SAS  System  16:20  Wednesday,  July  31,  1991 

Correlation  Analysis 

5  'VAR'  Variables:  INTERP11  INTERP14  INTERP19  INTERP2 2 

INTERP26 

Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

INTERP1 1 

4  5 

3.0444 

2 . 0555 

137 . 0000 

1 . 0000 

7.0000 

INTERP14 

45 

3.3444 

2 . 0666 

173 . 0000 

1 . 0000 

7 . 0000 

INTERP19 

4  5 

3.3333 

1 .9540 

150 . 0000 

1 . 0000 

7 . 0  0  G  0 

INTERP2  2 

45 

3 . 0889 

1.7687 

139 . 0000 

1 . 0000 

7 . 0000 

INTERP2  6 

4  5 

3 . 6667 

2.2259 

165 . 0000 

1 . 0000 

7 . 0000 

Correlation  Analysis 
Cronbach  Coefficient  Alpha 

for  RAW  variables  :  0.838012 

for  STANDARDIZED  variables:  0.839368 

Raw  Variables  Std .  Variables 

Deleted  Correlation  Correlation 

Variable _ with  Total _ Alpha _ with  Total _ Alpha 


INTERP1 1 
INTERP14 
INTERP19 
INTERP2? 
INTERP2  6 


0 . 439639 
0 .779779 
0. 732S99 
0 . 605942 
0 . 673753 


0.859153 
0 .764506 
0 .780176 
0 . 815508 
0.796517 


0. 436745 
0 . 775992 
0 .737340 
0 . 607411 
0.673819 


0.860771 
0.763936 
0 . 730167 
0 . 816447 
V  .  /  9  o  i  o  D 


Pearson  Correlation  Coefficients  /  Profc  >  1 R ;  under  Ho : 
Rho=0  /  N  =  45 


INTER  PI  1  INTER  PI  4  I  N'T  El 


I  N'TERP:. 1 


INTER? 11 
INTERP14 
INTERP19 
1NTERP22 


1 . ooouo 
0. 0 

0. 47248 
0. 0010 
0.38100 
0 . 0098 
0. 26769 
0 . 0754 
0  .  J  4  6  0  5 
0. 0199 


0  .  4  /  2 1  6 
0.0010 
1 . 00000 
0 . 0 

0.69414 
0 . 0001 
0 . 53861 
0 . 0001 
0 . 63018 
0 . 0001 


0.38100 
0 . 0098 
0 . 694  14 
0.0001 
1 .00000 
0.0 

0.62911 
0 . 0001 
0.58524 
0.0001 


0 . 2  o  7  6  9 
0.0754 
0.53861 
0.0001 
0.62911 
0 . 0  0  0  1 
1.00000 
0 . 0 

0.51571 
0 . 000  3 


0  .  34  o 0  5 
0.0199 
0 . 6  3  0  1 S 
0 . 000  1 
0 . 53524 
0 . 0  0  0  1 
0 . 5  1 5  7  1 
0 . 0  G  0  3 
1 . 00000 
0 . 0 


INTERP2  6 


Correlation  Analysis 

5  'VAR'  Variables:  COMM 13  COMM17  COMM24  COMM27  COMM31 


Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

COMM 13 

44 

3 . 8636 

1.8751 

170 . 0000 

1.0000 

7 . 0000 

COMM17 

45 

3 . 8222 

2 . 0033 

172 . 0000 

1 . 0000 

7 . 0000 

COMM24 

45 

4.3111 

2 . 054  1 

194 . 0000 

1 . 0000 

7 .0000 

COMM 2 7 

44 

3 .4091 

1 . 8964 

150 . 0000 

1 . 0000 

7 . 0000 

COMM31 

45 

3 .4222 

2 .2001 

154 . 0000 

1 . 0000 

7 . 0000 

Correlation  Analysis 
Cronbach  Coefficient  Alpha 

for  RAN  variables  :  0.838771 

for  STANDARDIZED  variables:  0.841553 

Raw  Variables  Std.  Variables 


Deleted  Correlation  Correlation 

Variable  with  Total _ Alpha _ with  Total  Alpha 


COMM 13 

0 . 6  5  4  2  6  6 

0.803506 

0  . 

6PPU5S  U.HOb^j4 

COMM17 

0 . 559069 

0.8284 

14 

0  . 

561813  0.831948 

COMM24 

0 . 648412 

0. 804303 

0  . 

644543  0.309811 

COMM27 

0.718137 

0. 736378 

0  . 

731508  0 .735579 

COMM3 1 

0.640926 

0.807669 

0  . 

639811  0.811101 

Pearson 

Correlation 

Coef  f ic i 

ents  / 

P 

rob  >  !R;  under  Ho: 

Rho=0  / 

Number  of 

Observations 

COMM 13 

COMM 17 

COMM  2 

4 

COMM 2 7  COMM  3  1 

COMM 13 

1 . 00000 

0 . 53809 

0.42499 

0.66153  0.46047 

0 . 0 

0 . 0002 

0 . 00 

•10 

0.0001  0.0017 

44 

4  4 

i  * 

4  3  4  4 

COMM 1 7 

0.53809 

1 . 00000 

0.40589 

0.45841  0 . 42995 

0. 0002 

0 . 0 

0 . 0057 

0.0018  0.0032 

4  4 

'*  D 

4  5 

4  *t  4  _3 

CO  MM  2  4 

0.42499 

0 .40589 

1 . 00000 

0.61738  0.60397 

0 . 0040 

0 . 0057 

0 . 0 

0.0001  0.0001 

.*  1 
*t  -t 

4  5 

4  5 

*14  *15 

CO MM 2 7 

0.66153 

0 . 4584  1 

0.61738 

1.00000  0.54526 

0.0001 

0.0018 

0 . 000  1 

0.0  0.0001 

/  > 

‘t  J 

4  4 

4  4 

4  4  4  4 

CO MM 3 1 

0.4604/ 

0 . 42995 

0 . 608 

97 

0.54526  1.00000 

0.0017 

0.0032 

0.0001 

0.0001  0.0 

4  4 

4  5 

45 

4  4  4  5 

9  5 


Correlation  Analysis 


5  'VAR'  Variables:  CREDIB10  CREDIB12  CREDIB20  CRED1B30 

CREDIB3  2 

Simple  Statistics 


Variable _ _N _ Mean _ Std  Dev  Sun _ Minimum  Maxi  nun 


CREDIB10 

45 

3.1333 

1 . 8902 

14  1 .0003 

1 . 0000 

7 . 0000 

CREDID12 

45 

3.3111 

1.7942 

149 .0000 

1 . 0000 

7 . 0000 

CREDIB2  0 

44 

3 . 5000 

2.0403 

154 . 0C00 

1  . OuOO 

7  .  0  0  U  0 

CREDIB3  0 

4  5 

3.2889 

1 .8043 

148 . 0900 

1.0000 

7.3000 

CREDIB3  2 

44 

3.4318 

1.8727 

151 . 0000 

1.0000 

7 . 0000 

Correlation 

Ana  1 y s i s 

Cronbach  Coefficient  Alpha 

for  RAW  variables  :  0.928477 

for  STANDARDIZED  variables:  0.929882 


Raw  Variables 
Deleted  Correlation 
Variable  with  Total _ Alpha 


Std.  Variables 
Correl.it  ion 
with  Total _ Aloha 


CREDIB10 

CREDIB12 

CREDIB20 

CREDIB30 

CREDIB32 


0.735427 

0.770196 

0.801601 

0.891098 

0.S71848 


0 . 9 26662 
0 .919944 
0.915127 
0.397449 
0 .900454 


0.737573 
0.769300 
0.800397 
0. 890338 
0.879634 


0 .923  3 3  9 
0.922372 
0.916583 
0.899137 
0.901247 


Pearson  Correlation  Coefficients  /  Prob  >  !R;  under  He: 
Rho~0  /  Number  of  Observations 


CREDI B 1 0 

CRHD I B1 2 

CREDI BOO 

CREDI  is 3  0 

CRT DIB  32 

CREDIB10 

1 . 00000 
0.0 

45 

0.56382 

0.0001 

4  5 

0.69546 

0 . 0001 

4  4 

0.7231 5 
0.0001 

4  5 

0  .  c  3  9  3  0 

0.00  0  1 

■  4 

*1  ■' 

CREDIB12 

0 . 56302 
0.0001 

4  5 

1.00000 

0 . 0 

•*  5 

0.67628 

0 . 0  0  0  i 

■1 

0  .  /  J  0  C’  i 

0  .  0  0  C  1 

>t  ‘J> 

0 . 7  8  2  2  0 

•t  -4 

CREDI B2 0 

0 . 69  5  4  6 

C.0U01 

0  .(>/()  J  o 
0.00  0  1 

1 . ooouo 

0 . 0 

0.757 U 2 

0 . 00  0  1 

U  .  7  3  4  9  4 

0 . JO  0  1 

CREDIB30 

0.72815 

0.000! 

4  5 

0 . 75087 

0 . 0  0  0  1 

4  5 

O . 75702 

0 . OOO 1 

‘i  *  1 

1 . 00000 

0 . 0 

“t  *J 

0-  .  o  o  \  3  / 

O  .  G  3  0  1 

‘t  ‘t 

CREDI B  3  2 

0 . 689  3  0 
Q.0U01 

4  4 

0.78220 

0 . 0  0  0  1 

4  4 

0.73454 

0. OuO 1 

4  3 

0  *  O  O  ‘1  J  ' 

0 . 0  0  0  1 

•t  -4 

j  .  0000' 0 

U  .  w 

1  * 

‘t  1 

Computations  for  Corrected  Measuring  Scale 
The  SAS  System  18:50  Monday,  August  5,  1991  1 

Correlation  Analysis 

b  'VAR'  Variables:  1NTERP11  INTERP14  1NTERP19  INTERP2  2 

INTERP2  6 


Simple  Statistics 


Variable 

N 

Mean 

Std  Dc 

v  Sum  Min 

ir-.um 

Max  i.T.um 

INTERP1 1 

24 

4 .2083 

1 . 7  jl8 9 

101.0000  2. 

0000 

7 . 0000 

INTERP1 4 

24 

5 .0000 

1.6940 

120.0000  2. 

0C  00 

/ . 0000 

INTERP 19 

24 

4.3333 

1.8  3  37 

104.0000  2. 

oooc 

7.0000 

INTERP22 

24 

3 .9583 

1.6545 

95.0000  2. 

0000 

7 . 0000 

INTERP26 

24 

4 .7083 

1 .9667 

113.0000  2. 

0000 

7 . 00 CO 

Correlation 

Analysis 

Cronbach  Coefficient  Alpha 

for  RAW  variables 

:  0.726921 

for  STANDARDIZED  variables:  0.7258 

*  r> 

‘i  O 

Raw  Va 

r  iables 

Std.  Va 

r  i  a  b 

les 

Deleted 

Correl ation 

Correlation 

Variable 

wi 

th  Total 

Aloha 

with  Total 

Ale 

ha 

INTERP1 1 

0. 

195208 

0 .781979 

0.193585 

0.782813 

INTERP 14 

0. 

693936 

0 . 599429 

0 . 687176 

0 . 595545 

INTERP 19 

0 . 

654093 

0. 609542 

0 . 6  5  5  8  9  x 

0 . 609110 

INTERP2  2 

0. 

432217 

0.700395 

0 . 430465 

0 . 7 

0  j 

INTERP26 

0. 

5  0  6  7  62 

0.673285 

0.503142 

0 . 6 

/  <L  i  i-  O 

Pearson  Correlation  Coefficients  /  Prcb  >  :R  under  lio: 
Rho=0  /  N  =  24 


INTERP 11 

INTERP 14 

INTERP 19 

INTERP2  2 

INTLRP26 

INTERl’l  j 

1  .  G  3  0  0  0 

0 . 3  j  o  5  G 

v)  .  U  0  *»  0  yj 

0. 05491 

■  \  ,  .  ■  ~  ' 

0. 0 

0.0355 

0.9330 

0 . 059  1 

•J  .  4  5  4  5 

INTERP 14 

O  .  3  5  8  3  6 

1.0000  0 

0  .  b  4  3  3  3 

0.31 0  2  o 

O  .  It  j  0 

0.0855 

0.0 

o 

o 

o 

o 

0.1401 

0.00 7 1 

INTERP 19 

0 , 004  60 

0 . 64338 

1.000 0  0 

0 . 62103 

0.51 1;  3  v 

0.93j0 

0. 0007 

o .  o 

0 . 0  0  1  2 

0  .  o  1  o  c 

I  NTEP.P2  2 

0 . 094  9  1 

C . 11026 

0 . 6  2  10  3 

1 . 00000 

0 .22  3  20 

0.6591 

C  .  1  4  0 1 

0.0012 

0 . 0 

0  .  5  5  4  3 

I NTERP2  6 

0 . 1602  3 

0 . 53503 

0 . 51039 

0.2232c 

1  •  v  J  b  0  u1 

0.454  5 

0 . 0071 

0 . 0103 

0.2043 

0 . 0 

Correlation  Analysis 


5  'VAR'  Variables:  COMM13  C0KM17  COMM24  COMM27  COMM 31 

Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

COMM13 

24 

4 . 5000 

1 . 6940 

108 . 0000 

2 .0000 

7 . 0000 

COMM17 

24 

4 . 1667 

1 . ^856 

100 . 0000 

2 .0000 

7 . 0000 

COMM24 

24 

5 . 0000 

1.8178 

120 . 0000 

2 . 0000 

7 . 0000 

COMM 2 7 

24 

3 .9583 

1 .7565 

95 . 0000 

2 .0000 

7 . 0000 

COMM 31 

24 

4 . 5000 

1 . 6631 

108 . 0000 

2 .0000 

7 . 0000 

Correlation  Analysis 
Crcr.bach  Coefficient  Alpha 

for  RAW  variables  :  0.399883 

for  STANDARDIZED  variables:  0.399590 

Raw  Variables  Std.  Variables 

Deleted  Correlation  Correlation 

Variable  with  Total _ A 1  pha _ with  Total  Aloha 


COMM 13 
COMM 17 
CO  MM  2  4 
COMM  2  7 
COMM  3  1 


0 . 768955 
0 . 741071 
0 . 819 107 
0.752276 
0 . 676869 


0.374220 

0.880209 

0.362421 

0.377651 

0.393371 


0 . 7644 62 
0 .741659 
0 . 822215 
0.753726 
0.673344 


0.374497 
0 . 879486 
0.361639 
0 . 876852 
0 . 894  137 


Pearson  Correlation  Coefficients  /  Prob  >  J  R !  under  Ho: 
Rho=0  /  N  =  24 


COMM  13 
COMM  1  7 
CCM.M24 
COMM 2  7 
CO  MM  3  1 


COMM 13 

1 . 00000 
0 . 0 

0 . 7  04  3  3 
0 . 000  1 
0.76246 
0 . 000  1 
0 . 7 03 ~  1 
0 . 000  1 
0 . 43082 
0.0356 


COMM  17 

0 .7  04 3  3 
0 . 000  1 
1 . 00000 
0 . 0 

0 . 62956 
0 . 0  0  1  0 
0 . 65386 
0 . 0005 
0 . 55463 
0 . 0049 


COMM 2 4 


0 . 76246 
0.0001 
0 . 62956 
0.0010 
1 . 00000 
0 . 0 

0.61273 
0.0015 
0 .75994 
0 . 000  1 


COMM  2  7 


0.70871 
0 . 0001 
0 . 65336 
0 . 0005 
0 . 61273 
0 . 0  0  1  6 
1 . 00000 
0  .  c 

0 . 60093 
0.0019 


COMM  3  1 


0 . 43082 
0.0356 
0  .  5  5  4  b  3 
0 . 0049 
0.75  9  9  4 
0  .  c o  1 
0  .  u  0  C  9  3 
0 . 0019 
1 . 00000 
0 . 0 


Correlation  Analysis 


5  'VAR'  Variables:  CREDIB10  CREDIB12  CREDIB2  0  CREDIB3  0 

CREDIB32 

Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

CREDIB10 

28 

3 . 8571 

1.6934 

108 . 0000 

2 . 0000 

7 . 0000 

CREDIB12 

28 

4 . 0357 

1. 6439 

113 . 0000 

2 . 0000 

7 . 0000 

CREDIB20 

28 

4 . 3929 

1 . 6852 

123 . 0000 

2 . 0000 

7 . 0000 

CREDIB30 

28 

4 . 1071 

1.6407 

115.0000 

2 . 0000 

7 . 0000 

CREDIB32 

28 

4 . 2  500 

1. 6694 

119 . 0000 

2 . 0000 

7 . 0000 

Correlation  Analysis 
Cronbach  Coefficient  Alpha 

for  RAW  variables  :  0.923670 

for  STANDARDIZED  variables:  0.923374 


Raw  Variables 
Correlation 
with  Total _ Alpha 


Deleted 

Variable 

CREDIB10 

CREDIB12 

CREDIB20 

CREDIB30 

CREDIB32 


0 . 699415 
0 . 760676 
0 . 806380 
0.846160 
0. 898949 


0.926279 
0.914 136 
0.905287 
0.397533 
0 .886618 


Std.  Variables 
Correlation 


with  Total 
0 . 699621 
0 .760362 
0. 806394 
0.846039 
0.899570 


r\  1  P  h  a 

0.926203 
0 .914603 
0 . 905513 
0 . 897733 
0 . 886880 


Pearson  Correlation  Coefficients  /  Prob  >  | R ]  under  Ho: 
Rho=0  /  N  =  28 


CREPT BIO  CREDIB12  CREDIB20  CREDIB30  CREPT  B3  2 


CREDIB10 

1 

. 

0 

0 

000 

0. 

5 

8729 

0 

• 

5 

9143 

0 

* 

6 

7 

£ 

24 

0 
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8 
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0 

• 

0 
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• 
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. 

0009 

0 

. 

c 

0 

01 

0 

. 

u 

*\ 

U 

n 

w 

i 

CREDIB12 

0 

• 

5 

8 

729 

1 . 

0 

0000 

0 

• 

7 

5680 

0 

. 

6 

5 

*7 

63 

0 

. 

*7 

i 

r > 
C 

9 

0 

. 

0 

010 

0 

. 

O 

0 

, 

0001 

0 

* 

0 

0 

01 

0 

, 

0 

0 

0 

1 

CREDIB20 

0 

. 

5 

9 

143 

0. 

7 

5630 

7 

J. 

. 

00000 

0 

. 

— T 
/ 

0 

/ 

57 

0 

. 

7 

9 

3 

1 

"7 

0 

• 

0 

009 

0 

. 

0GC1 

0 

, 

0 

0 

. 

0 

0 

01 

\J 

. 

0 

0 

0 

1 

CREDIB30 

0 

. 

6 

7 

224 

0  . 

6 

9  68 

0 

. 

7 

07  57 

1 

• 

0 

0 

0 

0  0 

0 

. 

q 

"'l 

<6 

2 

p 

o 

0 

. 

0 

001 

0 

. 

0001 

0 

. 

0001 

0 

. 

0 

0 

0 

n 

'9 

0 

i 

CREUIB32 

0 

. 

6 

8 

124 

0. 

/ 

1189 

0 

• 

7 

9317 

0 

. 

9 

2 

2 

8  8 

1 
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0 

0 
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001 

0 

. 

0001 

0 

. 

0001 

0 

0 

0 

01 

0 

0 

Appendix  D:  Median  Test  Results 


Legend : 


GRP- 


Sample  of  BEST  officers  1 
Sample  of  other  non-rated  officers  w/ops  2 
Sample  of  rated  officers  3 
Sample  of  officers  w/o  ops  4 
Supervisors  of  BEST  officers  5 
Supervisors  of  other  non-rated  officers  w/ops  6 
Supervisors  of  rated  officers  7 
Supervisors  of  officers  w/o  ops  3 
All  officers  with  operational  experience  9 


MEDIAN  TEST  FOR  QUESTION  11  =  GRP 

GRP 


. 

1 

2 

TOTAL 

ABOVE  MEDIAN 

5 

1 

6 

BELOW  MEDIAN 
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7 
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6 

7 

13 
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MEDIAN  TEST  FOR  QUESTION  14  =  GRP 

GRP 


1  2  TOT  A  I, 


ABOVE  MEDIAN  3  1 
BELOW  MEDIAN  2  5 
TOTAL  5  G 
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4 

11 
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MEDIAN  TEST  FOR  QUESTION  19  -  GRP 

GRP 
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MEDIAN  TEST  FOR  QUESTION 

22  =  GRP 
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MEDIAN  TEST  FOR  QUESTION 

26  =  GRP 
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/ 
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MEDIAN  TEST  FOR  QUESTION  13  =  GRP 


GRP 


_ 

1 

2 

TOTAL 

ABOVE  MEDIAN 

4 

3 

7 

BELOW  MEDIAN 

2 

3 

5 

TOTAL 

6 

6 

12 

TIES  WITH  MEDIAN 

2 

1 

3 

MEDIAN  VALUE  3  . 

CHI  SQUARE  0.34 

000 

DF  1 

P  VALUE 

0.5582 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 .  OE 

CASES  INCLUDED  15 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION  17  -  GRP 

GRP 


1  2 

TOTAL 

ABOVE  MEDIAN 

3  2 

5 

BELOW  MEDIAN 

2  3 

5 

TOTAL 

5  5 

10 

TIES  WITH  MEDIAN 

3  2 

5 

MEDIAN  VALUE  3  . 

MAX.  DIFF.  ALLOWED 

000 

BETWEEN  A 

TIE  1 

CASES  INCLUDED  15 

MISSING 

CASES  0 

MEDIAN  TEST  FOR  QUESTION  24  =  GRP 

GRP 


1 

2 

TOTAL 

ABOVE  MEDIAN 

5 

1 

6 

BELOW  MEDIAN 

2 

5 

7 

TOTAL 

7 

6 

13 

TIES  WITH  MEDIAN 

1 

1 

2 

MEDIAN  VALUE  5  . 

,  000 

CHI  SQUARE  3.90 

DF  1 

P  VALUE 

0.0483 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 0E 

CASES  INCLUDED  15 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  27  ^  GRP 

GRP 


1 

2 

TOTAL 

ABOVE  MEDIAN 

3 

1 

4 

BELOW  MEDIAN 

1 

5 

6 

TOTAL 

4 

6 

10 

TIES  WITH  MEDIAN 

3 

1 

4 

MEDIAN  VALUE  4 . 

MAX.  DIFF.  ALLOWED 

000 

BETWEEN 

A  TIE 

1 

CASES  INCLUDED  14 

MISSING  CASES 

1 

MEDIAN  TEST  FOR  QUESTION  31  =  GRP 

GRP 


1 

2 

TOTAL 

ABOVE  MEDIAN 

5 

0 

5 

BELOW  MEDIAN 

1 

6 

7 

TOTAL 

6 

6 

12 

TIES  WITH  MEDIAN 

2 

1 

3 

MEDIAN  VALUE  6. 

CHI  SQUARE  8.57 

000 

DF  1 

P  VALUE 

0.0034 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 

CASES  INCLUDED  15 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  10  =  GRP 

GRP 


1  2 

TOTAL 

ABOVE  MEDIAN 

5  0 

5 

BELOW  MEDIAN 

1  4 

5 

TOTAL 

6  4 

10 

TIES  WITH  MEDIAN 

2  3 

5 

MEDIAN  VALUE  2. 

MAX.  DIFF.  ALLOWED 

,  000 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  15 

MISSING  CASES 

0 
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MEDIAN  TEST  FOR  QUESTION  12 


GRP 


GRP 


1 

2 

TOTAL 

ABOVE  MEDIAN 

3 

3 

6 

BELOW  MEDIAN 

5 

2 

7 

TOTAL 

8 

5 

13 

TIES  WITH  MEDIAN 

0 

2 

2 

MEDIAN  VALUE  3. 

CHI  SQUARE  0.63 

000 

DF  1 

P  VALUE 

0.4285 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  15 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  20  =  GRP 

GRP 


_ 

1 

2 

TOTAL 

ABOVE  MEDIAN 

4 

3 

7 

BELOW  MEDIAN- 

2 

4 

6 

TOTAL 

6 

7 

13 

TIES  WITH  MEDIAN 

2 

0 

2 

MEDIAN  VALUE  2 . 

CHI  SQUARE  0.74 

000 

DF  1 

P  VALUE 

0 . 3906 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  15 

MISSING  CASES 

;  o 

MEDIAN  TEST  FOR  QUESTION  30  =  '  -P 

GRP 


. 

1 

2 

TOTAL 

ABOVE  MEDIAN 

5 

2 

7 

EELOW  MEDIAN 

2 

3 

5 

TOTAL 

7 

5 

12 

TIES  WITH  MEDIAN 

1 

2 

3 

MEDIAN  VALUE  2 . 

CHI  SQUARE  1.19 

000 

DF  1 

P  VALUE 

0.2763 

MAX .  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1  .  OE 

CASES  INCLUDED  15 

MISSING  CASES 

0 
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MEDIAN  TEST  FOR  QUESTION  32  =  GRP 

GRP 


1  2  TOTAL 

ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 
MEDIAN  VALUE  2.000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  14  MISSING  CASES  1 


MEDIAN  TEST  FOR  QUESTION  11  =  GRP 

GRP 


1  3  TOTAL 


ABOVE  MEDIAN  7 

BELOW  MEDIAN  1 

TOTAL  8 

TIES  WITH  MEDIAN  0 

MEDIAN  VALUE  4.500 

CHI  SQUARE  8.10  DF  1  P  VALUE  0.0044 
MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  18  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  14  =  GRP 

GRP 


1  3  TOTAL 

ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 
MEDIAN  VALUE  5.000 

CHI  SQUARE  1.73  DF  1  P  VALUE  0.188S 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  18  MISSING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION  19 


GRP 


GRP 


1 

3 

TOTAL 

- 

— 

ABOVE  MEDIAN 

4 

5 

9 

BELOW  MEDIAN 

4 

5 

9 

TOTAL 

8 

10 

18 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE  3. 

CHI  SQUARE  0.00 

500 

DF  1 

P  VALUE 

1.0000 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  18 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

22  =  GRP 

GRP 

1 

3 

TOTAL 

ABOVE  MEDIAN 

3 

4 

7 

BELOW  MEDIAN 

3 

4 

7 

TOTAL 

6 

8 

14 

TIES  WITH  MEDIAN 

2 

2 

4 

MEDIAN  VALUE  3 . 

000 

CHI  SQUARE  0.00 

DF  1 

P  VALUE 

1 . 0000 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 

CASES  INCLUDED  18 

MISS 

ING  CASES 

;  o 

MEDIAN  TEST  FOR  QUESTION 

26  =  GRP 

GRP 

1 

3 

TOTAL 

ABOVE  MEDIAN 

c; 

G 

BELOW  MEDIAN 

4 

3 

7 

TOTAL 

7 

n 

O 

15 

TIES  WITH  MEDIAN 

i 

2 

3 

MEDIAN  VALUE  5.000 

CHI  SQUARE  0.58  DF  1  P  VALUE  0.4468 
MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1. 
CASES  INCLUDED  18  MISSING  CASES  0 


0E-0005 


0E-0005 


0E-0005 
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MEDIAN  TEST  FOR  QUESTION  13  =  GRP 


GRP 

1 

3 

TOTAL 

ABOVE  MEDIAN 

2 

4 

6 

BELOW  MEDIAN 

4 

4 

8 

TOTAL 

6 

8 

14 

TIES  WITH  MEDIAN 

2 

2 

4 

MEDIAN  VALUE  5. 

000 

CHI  SQUARE  0.39 

DF  1  P 

VALUE 

0. 5329 

MAX.  DIFF.  ALLOWED 

BETWEEN  . 

A  TIE 

1. 0E-0005 

CASES  INCLUDED  18 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  17 

=  GRP 

GRP 

1 

3 

TOTAL 

ABOVE  MEDIAN 

3 

4 

7 

BELOW  MEDIAN 

5 

3 

8 

TOTAL 

8 

7 

15 

TIES  WITH  MEDIAN 

0 

3 

3 

MEDIAN  VALUE  5. 

000 

CHI  SQUARE  0.58 

DF  1  P 

VALUE 

0.4468 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 0E-0005 

CASES  INCLUDED  18 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  24 

=  GRP 

GRP 

1 

3 

TOTAL 

ABOVE  MEDIAN 

5 

3 

3 

BELOW  MEDIAN 

3 

C 

3 

TOTAL 

8 

3 

16 

TIES  WITH  MEDIAN 

0 

2 

S. 

MEDIAN  VALUE  6 . 

000 

CHI  SQUARE  1.00 

DF  1  P 

VALUE 

0.3173 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . OE-0005 

CASES  INCLUDED  13 

MISSING  CASES 

0 
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MEDIAN  TEST  FOR  QUESTION27  =  GRP 


GRP 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 

MEDIAN  VALUE 
CHI  SQUARE  0.69 
MAX.  DIFF.  ALLOWEI 
CASES  INCLUDED  17 


above  median 

BELOW  MEDIAN 
TOTAL 

TIES  WITH  median 

median  value 
CHI  SQUARE  8.10 
MAX.  DIFF.  ALLOWEI 
CASES  INCLUDED  18 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 


median  value 

CHI  SQUARE  0.58 
MAX.  DIFF.  ALLOWED  BETWEEN 
CASES  INCLUDED  18 


1 

3 

TOTAL 

3 

4 

7 

1 

4 

5 

4 

8 

12 

3 

2 

5 

.  000 

DF  I 

P  VALUE 

0.4076 

i  BETWEEN  A  TIE 

l.i 

MISSING  CASES 

1 

'ESTION 

31  =  GRP 

GRP 

1 

3 

TOTAL 

7 

2 

9 

1 

3 

9 

8 

10 

13 

0 

0 

0 

.  500 

DF  1 

P  VALUE 

0 . 0044 

)  BETWEEN  A  TIE 

1 . 

MISSING  CASES 

;  o 

IESTION 

10  =  GRP 

GRP 

1 

3 

TOTAL 

4 

3 

7 

3 

5 

8 

/ 

8 

15 

1 

2 

3 

1 . 000 

DF  1 

P  VALUE 

0.4468 

A  TIE 
MISSING  CASES 


1 . OE-0005 
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MEDIAN  TEST  FOR  QUESTION  12  =  GRP 

GRP 


I  3  TOTAL 

ABOVE  MEDIAN  36  9 

BELOW  MEDIAN  54  9 

TOTAL  8  10  18 

TIES  WITH  MEDIAN  00  0 

MEDIAN  VALUE  3.500 

CHI  SQUARE  0.90  DF  1  P  VALUE  0.3428 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  18  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  20  =  GRP 

GRP 


1  3  TOTAL 

ABOVE  MEDIAN  45  9 

BELOW  MEDIAN  45  9 

TOTAL  6  10  18 

TIES  WITH  MEDIAN  00  0 

MEDIAN  VALUE  3.500 

CHI  SQUARE  0.00  DF  1  P  VALUE  1.0000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  18  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  30  =  GRP 

GRP 


1  3  TOTAL 

ABOVE  MEDIAN 
BELOW  MEDIAN- 
TOTAL 

TIES  WITH  MEDIAN- 
MEDIAN  VALUE  4.000 

CHI  SQUARE  0.03  DF  1  P  VALUE  0.7321 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  18  MISSING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION  32 


GRP 


GRP 


l 

l 

m  1 

1 

1 

1 

1 

*H  1 

1 

TOTAL 

ABOVE  MEDIAN 

3  5 

8 

BELOW  MEDIAN 

2  3 

5 

TOTAL 

S  8 

13 

TIES  WITH  MEDIAN 

2  2 

4 

MEDIAN  VALUE  4 . 

CHI  SQUARE  0.01 

000 

DF  1  P  VALUE 

0 .9282 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1  . 

CASES  INCLUDED  17 

MISSING  CASES 

i  1 

MEDIAN  TEST  FOR  QUESTION  11  =  GRP 

GRP 


1 

*-f 

TOTAL 

ABOVE  MEDIAN 

7 

6 

13 

BELOW  MEDIAN 

1 

11 

12 

TOTAL 

8 

17 

2  0 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE 

CHI  SQUARE  5.04 

000 

DF  1 

P  VALUE 

0.0148 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1  . 

CASES  INCLUDED  29  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  14  =  GRP 

GRP 


1  4  TOTAL 

ABOVE  MEDIAN  3 

BELOW  MEDIAN  2 

TOTAL  5 

TIES  WITH  MEDIAN  3 

MEDIAN  VALUE  4.000 

CHI  SQUARE  0.87  DF  1  P  VALUE  U.3500 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1 . 0E-0005 

CASES  INCLUDED  29  MILLING  CASES  0 


7 

12 

19 

9 


10 


MEDIAN  TEST  FOR  QUESTION  19 


GRP 


GRP 


1 

4 

TOTAL 

ABOVE  MEDIAN 

4 

10 

14 

BELOW  MEDIAN 

2 

8 

10 

TOTAL 

6 

1  8 

24 

TIES  WITH  MEDIAN 

2 

3 

5 

MEDIAN  VALUE  3  . 

CHI  SQUARE  0.23 

000 

DF  1 

P  VALUE 

0.6326 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1  . 

CASES  INCLUDED  29 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  22  =  GRP 

GRP 


ABOVE  MEDIAN  3 

BELOW  MEDIAN  3 

(TiArn  R  t  /- 

IQinL  O 

TIES  WITH  MEDIAN  2 

MEDIAN  VALUE  3.000 

CHI  SQUARE  0.06  DF  1  P  VALUE  0.8130 

MAX  DIFF.  ALLOWED  BETWEEN  A  TIE  1 . OE-0005 

CASES  INCLUDED  29  MISSING  CASES  0 


TnTl T  . 


10 

8 


13 

11 

4 

5 


MEDIAN  TEST  FOR  QUESTION  26  =  GRP 

GRP 


1  4  TOTAL 

ABOVE  MEDIAN  5 

BELOW  MEDIAN  1 

TOTAL  G 

TIES  WITH  MEDIAN  2 

MEDIAN  VALUE  3.000 

CHI  SQUARE  3.56  DF  1  P  VALUE  0.0593 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1 . 0E-0G05 

CASES  INCLUDED  29  MISSING  CASES  0 


7  12 

1  1  1 
18  24 

3  5 


1  1  1 


MEDIAN  TEST  FOR  QUESTION  13 


GRP 


GRP 


1 

4 

TOTAL 

- 

- — 

ABOVE  MEDIAN 

4 

9 

13 

BELOW  MEDIAN 

2 

7 

9 

TOTAL 

6 

16 

22 

TIES  WITH  MEDIAN 

2 

3 

5 

MEDIAN  VALUE  3  . 

CHI  SQUARE  0.20 

000 

DF  1 

P  VALUE 

0.6581 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 0E-0005 

CASES  INCLUDED  27 

MISSING  CASES 

2 

MEDIAN  TEST  FOR  QUESTION  17  =  GRP 

GRP 

1  4 

TOTAL 

ABOVE  MEDIAN 

3  9 

12 

BELOW  MEDIAN 

2  7 

9 

TOTAL 

5  16 

21 

TIES  WITH  MEDIAN 

3  5 

8 

MEDIAN  VALUE  3 

CHI  SQUARE  0.02 

.  000 

DF  1  P  VALUE 

0.8824 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 0E-0005 

CASES  INCLUDED  29 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  24  =  GRP 

GRP 

1  4 

ABOVE  MEDIAN  6  7 

TOTAL 

13 

BELOW  MEDIAN 

1  10 

1 1 

TOTAL 

7  17 

2  4 

TIES  WITH  MEDIAN 

1  4 

5 

MEDIAN  VALUE  4 

CHI  SQUARE  3.96 

.  000 

DF  1  P  VALUE 

0.0465 

MAX.  DIFf.  ALLOWED 

BETWEEN  A  TIE 

1  . 

CASES  INCLUDED  29  MISSING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION27  =  GRP 


GRP 


1 

4 

TOTAL 

ABOVE  MEDIAN 

6 

7 

13 

BELOW  MEDIAN 

1 

10 

11 

TOTAL 

7 

17 

24 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE 

3 . 000 

CHI  SQUARE  3.96 

DF  1  1 

P  VALUE 

0.0465 

MAX.  DIFF.  ALLOWED  BETWEEN 

A  TIE 

1.  ( 

CASES  INCLUDED  28 

MISSING  CASES 

1 

MEDIAN  TEST  FOR  QUESTION  31  =  GRP 

GRP 

1 

4 

TOTAL 

ABOVE  MEDIAN 

7 

6 

13 

BELOW  MEDIAN 

0 

10 

10 

TOTAL 

7 

16 

23 

TIES  WITH  MEDIAN 

1 

5 

6 

MEDIAN  VALUE 

3 . 000 

CHI  SQUARE  7.74 

DF  1 

P  VALUE 

0.0054 

MAX.  DIFF.  ALLOWED  BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  29 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION  10  =  GRP 

GRP 

1 

4 

TOTAL 

ABOVE  MEDIAN 

3 

3 

1 1 

BELOW  MEDIAN 

4 

10 

14 

TOTAL 

7 

13 

2  D 

TIES  WITH  MEDIAN 

i 

3 

4 

MEDIAN  VALUE 

4 . 000 

CHI  SQUARE  0.01 

DF  1 

P  VALUE 

0.9423 

MAX.  DIFF.  ALLOWED  BETWEEN 

A  TIE 

1  . 

0E-0005 


CASES  INCLUDED  29 


MISSING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION  12  =  GRP 

GRP 


1 

4 

TOTAL 

ABOVE  MEDIAN 

3 

9 

12 

BELOW  MEDIAN 

1 

5 

6 

TOTAL 

4 

14 

13 

TIES  WITH  MEDIAN 

4 

7 

11 

MEDIAN  VALUE 

2 . 000 

CHI  SQUARE  0.16 

DF  1 

P  VALUE 

0 . 6835 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 

1 . ' 

CASES  INCLUDED  29 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

20  =  GRP 

GRP 

1 

4 

TOTAL 

ABOVE  MEDIAN 

2 

7 

9 

BELOW  MEDIAN 

4 

9 

13 

TOTAL 

6 

16 

22 

TIES  WITH  MEDIAN 

2 

3 

5 

MEDIAN  VALUE 

4 . 000 

CHI  SQUARE  0.20 

DF  1 

P  VALUE 

0.6581 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 

1 . 

CASES  INCLUDED  27 

MISSING  CASES 

i  2 

MEDIAN  TEST  FOR  QUESTION 

30  =  GRP 

GRP 

1 

4 

TOTAL 

ABOVE  MEDIAN 

4 

7 

11 

BELOW  MEDIAN 

*> 

8 

1  1 

TOTAL 

7 

■>  r 

J.  -) 

-» 

t-. 

TIES  WITH  MEDIAN 

i 

g 

7 

MEDIAN  VALUE 

3 . 000 

CHI  SQUARE  0.21 

DF  1 

P  VALUE 

0.6471 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 
CASES  INCLUDED  29  MISSING  CASES  0 


1 . 0E-0005 


13  4 


MEDIAN  TEST  FOR  QUESTION  32  =  GRP 

GRP 


1 

4 

TOTAL 

ABOVE  MEDIAN 

5 

8 

13 

BELOW  MEDIAN 

2 

11 

13 

TOTAL 

7 

19 

26 

TIES  WITH  MEDIAN 

0 

2 

2 

MEDIAN  VALUE  3 

.  000 

CHI  SQUARE  1.76 

DF  1 

P  VALUE 

0. 184: 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  28 

MISS 

ING  CASES 

1 

MEDIAN  TEST  FOR  QUESTION 

11  =  GRP 

GP 

p 

3 

4 

TOTAL 

ABOVE  MEDIAN 

5 

10 

15 

BELOW  MEDIAN 

5 

7 

12 

TOTAL 

10 

17 

27 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE  2 

.  000 

CHI  SQUARE  0.20 

DF  1 

P  VALUE 

0.655' 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  31 

MISS 

ING  CASES  C 

MEDIAN  TEST  FOR  QUESTION 

14  =  GRP 

- 

GRP 

3 

A 

4 

TOTAL 

ABOVE  MEDIAN 

8 

7 

15 

BELOW  MEDIAN 

1 

12 

1  3 

TOTAL 

9 

19 

28 

TIES  WITH  MEDIAN 

1 

2 

3 

0E-0005 


MEDIAN  VALUE  4.000 

CHI  SQUARE  6.65  DF  1 
MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 
CASES  INCLUDED  31  MISSING  CASES 


P  VALUE  0.0099 

1 . 0E-0005 

0 
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MEDIAN  TEST  FOR  QUESTION  19  =  GRP 

GRP 


3  4  TOTAL 


ABOVE  MEDIAN 

5 

10 

15 

BELOW  MEDIAN 

4 

8 

12 

TOTAL 

9 

IS 

27 

TIES  WITH  MEDIAN 

1 

3 

4 

MEDIAN  VALUE  3.000 

CHI  SQUARE  C.00  DF  1  P  VALUE  1.0000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  31  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  22  =  GRP 


GRP 


3 

4  TOTAL 

ABOVE  MEDIAN 

4 

10  14 

BELOW  MEDIAN 

4 

8  3  2 

TOTAL 

8 

18  26 

TIES  WITH  MEDIAN 

2 

3  5 

MEDIAN  VALUE  3. 

000 

CHI  SQUARE  0.07 

DF  1 

P  VALUE  0.7931 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE  1.0E-0005 

CASES  INCLUDED  31 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION  26  =  GRP 

GRP 

3  4 

TOTAL 

ABOVE  MEDIAN 

8  7 

15 

BELOW  MEDIAN 

1  1  i 

12 

TOTAL 

CO 

rH 

27 

TIES  WITH  MEDIAN 

1  3 

MEDIAN  VALUE  3 . 

000 

CHI  SQUARE  6.08 

DF  1  P  VALUE 

0.0137 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 0E-0005 

CASES  INCLUDED  31 

MISSING  CASES 

0 

116 


MEDIAN  TEST  FOR  QUESTION  13 


GRP 


GRP 


3 

4 

TOTAL 

ABOVE  MEDIAN 

6 

8 

14 

BELOW  MEDIAN 

3 

10 

13 

TOTAL 

9 

18 

27 

TIES  WITH  MEDIAN 

1 

1 

2 

MEDIAN  VALUE  4 . 

CHI  SQUARE  1.19 

000 

DF  1 

P  VALUE 

0.2760 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 0E-0005 

CASES  INCLUDED  29 

MISSING  CASES 

;  2 

MEDIAN  TEST  FOR  QUESTIONS  17  =  GRP 

GRP 


3 

4 

TOTAL 

ABOVE  MEDIAN 

7 

8 

15 

BELOW  MEDIAN 

1 

12 

13 

TOTAL 

8 

20 

23 

TIES  WITH  MEDIAN 

2 

1 

3 

MEDIAN  VALUE  4 . 

CHI  SQUARE  5.18 

000 

DF  1 

P  VALUE 

0. 0228 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1  .  OE 

CASES  INCLUDED  31 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  24  =  GRP 

GRP 


3 

4 

TOTAL 

ABOVE  MEDIAN 

6 

7 

13 

BELOW  MEDIAN 

10 

13 

TOTAL 

9 

17 

26 

TIES  WITH  MEDIAN 

1 

4 

5 

MEDIAN  VALUE  4. 

000 

CHI  SQUARE  1.53 

DF  1 

P  VALUE 

0 .2162 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1  .  OE 

CASES  INCLUDED  31  MISSING  CASES  0 


117 


MEDIAN  TEST  FOR  QUESTION27  =  GRP 


GRP 


3 

4 

TOTAL 

ABOVE  MEDIAN 

6 

7 

13 

BELOW  MEDIAN 

4 

10 

14 

TOTAL 

10 

17 

27 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE  3 . 

.  000 

CHI  SQUARE  0.89 

DF  1 

P  VALUE 

0. 3445 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 0E- 

CASES  INCLUDED  31 

MISSING  CASES 

1  0 

MEDIAN  TEST  FOR  QUESTION 

31  =  GRP 

GRP 

3 

4 

TOTAL 

ABOVE  MEDIAN 

6 

6 

12 

BELOW  MEDIAN 

4 

10 

14 

TOTAL 

10 

16 

26 

TIES  WITH  MEDIAN 

0 

5 

5 

MEDIAN  VALUE  3 

.  000 

CHI  SQUARE  1.25 

DF  1 

P  VALUE 

0.2629 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 .  OE 

CASES  INCLUDED  31 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION 

10  =  GRP 

G 

RP 

3 

4 

TOTAL 

ABOVE  MEDIAN 

3 

11 

14 

BELOW  MEDIAN 

5 

9 

14 

TOTAL 

8 

20 

28 

TIES  WITH  MEDIAN 

2 

1 

3 

MEDIAN  VALUE  3 

.  000 

CHI  SQUARE  0.70 

DF  1 

P  VALUE 

0.4028 

MAX.  DIFF.  ALLOWED  BETWEEN  A 


CASES  INCLUDED  31 


MISSINC 


TIE 

CASES 


1 . OE-0005 


1.18 


MEDIAN  TEST  FOR  QUESTION  12 


GRP 


GRP 


3 

4 

TOTAL 

ABOVE  MEDIAN 

6 

8 

14 

BELOW  MEDIAN 

2 

12 

14 

TOTAL 

8 

20 

28 

TIES  WITH  MEDIAN 

2 

1 

3 

MEDIAN  VALUE  3. 

000 

CHI  SQUARE  2.80 

DF  1 

P  VALUE 

0. 0943 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

l.i 

CASES  INCLUDED  31 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

20  =  GRP 

GRP 

3 

4 

TOTAL 

ABOVE  MEDIAN 

5 

7 

12 

BELOW  MEDIAN 

5 

9 

14 

TOTAL 

10 

16 

26 

TIES  WITH  MEDIAN 

0 

3 

3 

MEDIAN  VALUE  4. 

000 

CHI  SQUARE  0.10 

DF  1 

P  VALUE 

0.7558 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  29 

MISSING  CASES 

;  2 

MEDIAN  TEST  FOR  QUESTION 

30  =  GRP 

- 

GRP 

-> 

_/ 

4 

TOTAL 

ABOVE  MEDIAN 

6 

7 

13 

BELOW  MEDIAN 

1 

8 

9 

TOTAL 

7 

15 

22 

TIES  WITH  MEDIAN 

3 

6 

9 

MEDIAN  VALUE  3. 

000 

CHI  SQUARE  3.01 

DF  1 

P  VALUE 

0 . 0827 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  31 


MISSING  CASES  0 


119 


MEDIAN  TEST  FOR  QUESTION  32  =  GRP 


GRP 


_ 

3 

4 

TOTAL 

ABOVE  MEDIAN 

7 

8 

15 

BELOW  MEDIAN 

1 

11 

12 

TOTAL 

8 

19 

27 

TIES  WITH  MEDIAN 

2 

2 

4 

MEDIAN  VALUE  3. 

CHI  SQUARE  4.70 

000 

DF  1 

P  VALUE 

0.0302 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 

CASES  INCLUDED  31 

MISSING  CASES 

;  o 

MEDIAN  TEST  FOR  QUESTION  11  ^  GRP 

GRP 


_ 

5  b 

TOTAL 

ABOVE  MEDIAN 

2  3 

5 

BELOW  MEDIAN 

2  2 

* 

TOTAL 

4  5 

9 

TIES  WITH  MEDIAN 

0  2 

2 

MEDIAN  VALUE  3 . 

MAX.  DIFF.  ALLOWED 

000 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  11 

MISSING  CASES 

1 

MEDIAN  TEST  FOR  QUESTION  14  =  GRP 

GRP 


5  6  TOTAL 

ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 
MEDIAN  VALUE  3.500 

CHI  SQUARE  3.09  DF  1  P  VALUE  0.0790 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


120 


MEDIAN  TEST  FOR  QUESTION  19  =  GRP 

GRP 


5  6 

TOTAL 

ABOVE  MEDIAN 

1  3 

4 

BELOW  MEDIAN 

2  2 

4 

TOTAL 

3  5 

8 

TIES  WITH  MEDIAN 

2  2 

4 

MEDIAN  VALUE  4  . 

MAX.  DIFF.  ALLOWED 

000 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  22  =  GRP 

GRP 


5 

6  TOTAL 

ABOVE  MEDIAN 

1 

5  6 

BELOW  MEDIAN 

4 

2  6 

TOTAL 

5 

7  12 

TIES  WITH  MEDIAN 

0 

0  0 

MEDIAN  VALUE  3 . 

CHI  SQUARE  3 . 09 

500 

DF  1 

P  VALUE  0.0790 

MAX.  DIFF.  ALLOWED 

between 

A  TIE  1.0E 

CASES  INCLUDED  12 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION  26  -  GRP 

GRP 


_ 

5 

6 

TOTAL 

ABOVE  MEDIAN 

1 

5 

6 

BELOW  MEDIAN 

2 

6 

TOTAL 

5 

7 

12 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE  3. 

CHI  SQUARE  3.09 

500 

DF  1 

P  VALUE 

0 . 0790 

MAX.  DIFF.  ALLOWED 

between 

A  TIE 

1  . 

CASES  INCLUDED  12 

MISSING  CASES 

0 
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MEDIAN  TEST  FOR  QUESTION  13 


GRP 


GRP 


5  6 

TOTAL 

ABOVE  MEDIAN 

2  2 

4 

BELOW  MEDIAN 

2  3 

5 

TOTAL 

4  5 

9 

TIES  WITH  MEDIAN 

1  2 

3 

MEDIAN  VALUE  4. 

000 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 0E-0005 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  17  =  GRP 

GRP 


5 

6 

TOTAL 

ABOVE  MEDIAN 

2 

4 

6 

BELOW  MEDIAN 

3 

3 

6 

TOTAL 

5 

7 

12 

TIES  WITH  MEDIAN 

0 

0 

r\ 

U 

MEDIAN  VALUE  4 

.  500 

CHI  SQUARE  0.34 

DF  1  P 

VALUE 

0 . 5532 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 0E-0005 

CASES  INCLUDED  12 

MISSIN 

G  CASES 

0 

MEDIAN  TEST  FOR 

QUESTION  24  =  GRP 

GRP 

5 

6 

TOTAL 

A30VE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 

2 

2 

3 

2 

5 

2 

5 

9 

3 

MEDIAN  VALUE  5.000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


122 


MEDIAN  TEST  FOR  QUESTION27  =  GRP 

GRP 


5  6  TOTAL 

ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 
MEDIAN  VALUE  4.000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  31  =  GRP 

GRP 


5  6  TOTAL 

ABOVE  MEDIAN  2  3 

BELOW  MEDIAN  3  2 

TOTAL  5  5 

TIES  WITH  MEDIAN  0  2 

MEDIAN  VALUE  4.000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  10  =  GRP 

GRP 


5  6  *  TOTAL 

ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 
MEDIAN  VALUE  2.500 

CHI  SQUARE  3.09  DF  1  P  VALUE  0.0790 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1 . 0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


5 

5 

10 

2 


123 


MEDIAN  TEST  FOR  QUESTION  12  =  GRP 


GRP 


5 

6 

TOTAL 

above  median 

1 

4 

5 

BELOW  MEDIAN 

3 

2 

5 

TOTAL 

4 

6 

10 

TIES  WITH  MEDIAN 

1 

1 

2 

MEDIAN  VALUE  4 

.  000 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

20  =  GRP 

GRP 

5 

6 

TOTAL 

ABOVE  MEDIAN 

1 

3 

4 

BELOW  MEDIAN 

3 

2 

5 

TOTAL 

4 

5 

9 

TIES  WITH  MEDIAN 

i 

2 

-» 

J 

MEDIAN  VALUE  4 

.  000 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  12 

MIS 

SING  CASES 

;  o 

MEDIAN  TEST  FOR  QUESTION 

30  =  GRP 

GRP 

-5 

6 

'TOTAL 

ABOVE  MEDIAN 

1 

4 

5 

BELOW  MEDIAN 

4 

1 

TOTAL 

5 

5 

10 

TIES  WITH  MEDIAN 

0 

2 

1 . 0E-00C5 


1 . QE-0005 


MEDIAN  VALUE  4.000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


12  4 


MEDI/  I  TEST  FOR  QUESTION  32 


GRP 


GRP 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 

MEDIAN  VALUE 
MAX.  DIFF.  ALLOWEI 
CASES  INCLUDED  12 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN- 

MEDIAN  VALUE 
MAX.  DIFF.  ALLOWEI 
CASES  INCLUDED  11 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 


5 

6 

TOTAL 

1 

4 

5 

2 

1 

3 

3 

5 

8 

2 

2 

4 

.  000 

BETWEEN 

A  TIE 

1 , 

MISSING  CASES 

0 

ESTION  11 

=  GRP 

GRP 

5 

7 

TOTAL 

2 

2 

4 

0 

C 

0 

2 

2 

4 

2 

5 

7 

000 

)  BETWEEN 

A  TIE 

i 

MISSING  CASES 

;  l 

1  ESTION  14 

=  GRP 

GRP 

5 

7 

TOTAL 

1 

b 

6 

4 

2 

6 

5 

7 

12 

o 

o 

ij 

1 . 0E-  0005 


1 . 0E-0005 


MEDIAN  VALUE  3.500 

CHI  SQUARE  3.09  DF  1  P  VALUE  0.0790 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


12  5 


MEDIAN  TEST  FOR  QUESTION  19 


GRP 


GRP 


_ 

5  7 

TOTAL 

ABOVE  MEDIAN 

1  3 

4 

BELOW  MEDIAN 

2  1 

3 

TOTAL 

3  4 

7 

TIES  WITH  MEDIAN 

2  3 

5 

MEDIAN  VALUE  4. 

MAX.  DIFF.  ALLOWED 

000 

BETWEEN  A  TIE 

1 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  22  =  GRP 

GRP 


5  7  TOTAL 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIES  WITH  MEDIAN 


2  4  6 

3  3  6 

5  7  12 

0  0  0 


MEDIAN  VALUE  2.500 

CHI  SQUARE  0.34  DF  1  P  VALUE  0.5582 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  26  =  GRP 

GRP 


5  7  TOTAL 


ABOVE  MEDIAN 
BELOW  MEDIAN 
TOTAL 

TIFS  WITH  MEDIAN 


14  5 

3  2  5 

4  6  10 

11  2 


MEDIAN  VALUE  3.000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  12  MISSING  CASES  0 


126 


MEDIAN  TEST  FOR  QUESTION  13 


GRP 


GRP 


5 

7 

TOTAL 

ABOVE  MEDIAN 

2 

4 

6 

BELOW  MEDIAN 

3 

3 

6 

TOTAL 

5 

7 

12 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE  4 . 

500 

CHI  SQUARE  0.34 

DF  1 

P  VALUE 

0.5582 

MAX.  DIFF.  ALLOWED 

3ETWEEN  A  TIE 

1 . 

CASES  INCLUDED  12 

MISSING  CASES 

3 

MEDIAN  TEST  FOR  QUESTION 

17  =  GRP 

GRP 

5 

7 

TOTAL 

ABOVE  MEDIAN 

2 

4 

6 

BELOW  MEDIAN 

3 

3 

6 

TOTAL 

5- 

7 

12 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE  5. 

500 

CHI  SQUARE  0.34 

DF  1 

P  VALUE 

0.5582 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

24  =  GRP 

G 

RP 

5 

7 

TOTAL 

ABOVE  MEDIAN 

2 

2 

4 

BELOW  MEDIAN 

2 

2 

4 

TOTAL 

Hr 

4 

3 

TIES  WITH  MEDIAN 

l 

3 

4 

MEDIAN  VALUE  5. 

000 

MAX.  DIFF.  ALLOWED 

BET  WE 

EN  A  TIE 

1  . 

CASES  INCLUDED  12 

MIS 

SING  CASES 

;  o 

127 

i .  oe-ooo: 


MEDIAN  TEST  FOR  QUESTION27  =  GRP 

GRP 


5 

7 

TOTAL 

ABOVE  MEDIAN 

1 

5 

6 

BELOW  MEDIAN 

4 

2 

6 

TOTAL 

5 

7 

12 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE 

3 . 500 

CHI  SQUARE  3.09 

DF  1 

P  VALUE 

0 . 0791 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 

1 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

31  =  GRP 

GRP 

5 

"7 

TOTAL 

ABOVE  MEDIAN 

3 

2 

5 

BELOW  MEDIAN 

0 

4 

4 

TOTAL 

3 

6 

9 

TIES  WITH  MEDIAN 

2 

1 

3 

MEDIAN  VALUE 

2 . 000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 

1 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

10  =  GRP 

GRP 

5 

7 

TOTAL 

ABOVE  MEDIAN 

1 

4 

5 

BELOW  MEDIAN 

3 

2 

5 

TOTAL 

4 

G 

10 

TIES  WITH  MEDIAN 

i 

i 

2 

MEDIAN  VALUE 

2 . 000 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 

1 

1 . 0E-0005 


1 . 0E-0005 


CASES  INCLUDED  12 


MISSING  CASES  0 


128 


MEDIAN  TEST  FOR  QUESTION  12 


GRP 


GRP 


_ 

5 

7 

TOTAL 

ABOVE  MEDIAN 

1 

3 

4 

BELOW  MEDIAN 

3 

2 

5 

TOTAL 

4 

5 

9 

TIES  WITH  MEDIAN 

1 

2 

3 

MEDIAN  VALUE  4  . 

MAX.  DIFF.  ALLOWED 

000 

BETWEEN 

A  TIE 

1 

CASES  INCLUDED  12 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  20  =  GRP 

GRP 


5 

7 

TOTAL 

ABOVE  MEDIAN 

1 

4 

C 

BELOW  MEDIAN 

3 

2 

TOTAL 

4 

6 

10 

TIES  WITH  MEDIAN 

1 

1 

2 

MEDIAN  VALUE  4. 

000 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 

CASES  INCLUDED  12  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  30  =  GRP 

GRP 


5 

/ 

TOTAL 

ABOVE  MEDIAN 

1 

5 

6 

BELOW  MEDIAN 

4 

2 

6 

TOTAL 

5 

7 

12 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE  3. 

CHI  SQUARE  3.09 

900 

DF  1  P 

1  VALUE 

0.0790 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1  .  OE 

CASES  INCLUDED  12  MISSING  CASES  0 


129 


MEDIAN  TEST  FOR  QUESTION  32  =  GRP 

GRP 


5  7 

TOTAL 

ABOVE  MEDIAN 

1  3 

4 

BELOW  MEDIAN 

2  3 

5 

TOTAL 

3  6 

9 

TIES  WITH  MEDIAN 

2  0 

2 

MEDIAN  VALUE  4 . 

000 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  11 

MISSING  CASES 

1 

MEDIAN  TEST  FOR  QUESTION  11  =  GRP 

GRP 

5  8 

TOTAL 

ABOVE  MEDIAN 

2  7 

9 

BELOW  MEDIAN 

2  6 

8 

TOTAL 

4  13 

17 

TIES  WITH  MEDIAN 

0  3 

3 

MEDIAN  VALUE  3 . 

000  • 

CHI  SQUARE  0.C2 

DF  1  P  VALUE 

0.8923 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  20 

MISSING  CASES 

1 

MEDIAN  TEST  FOR  QUESTION  14  =  GRP 

GRP 

5  8 

TOTAL 

ABOVE  MEDIAN 

1  8 

9 

BELOW  MEDIAN 

4  5 

9 

TOTAL 

5  13 

18 

TIES  WITH  MEDIAN 

0  3 

3 

MEDIAN  VALUE  4 . 

000 

CWT  SQUARE  2.49 

DF  1  P  VALUE 

0.1144 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1  . 

CASES  INCLUDED  21 

MISSING  CASES 

;  o 

1.0E-0005 


130 


MEDIAN  TEST  FOR  QUESTION  19  =  GRP 


GRP 


5 

8 

TOTAL 

ABOVE  MEDIAN 

1 

8 

9 

BELOW  MEDIAN 

2 

2 

4 

TOTAL 

3 

10 

13 

TIES  WITH  MEDIAN 

2 

6 

8 

MEDIAN  VALUE  4 . 

CHI  SQUARE  2.36 

000 

DF  1 

P  VALUE 

0 . 1245 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  21 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  22  =  GRP 

GRP 


__ 

5 

8 

TOTAL 

ABOVE  MEDIAN 

1 

7 

8 

BELOW  MEDIAN 

4 

5 

9 

TOTAL 

5 

12 

17 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE  4. 

CHI  SQUARE  2.08 

000 

DF  1 

P  VALUE 

0.1491 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  21 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  26  =  GRP 

GRP 


_ 

5  3 

TOTAL 

ABOVE  MEDIAN 

0  9 

9 

BELOW  MEDIAN 

4  4 

3 

TOTAL 

4  13 

17 

TIES  WITH  MEDIAN 

1  3 

MEDIAN  VALUE  4 . 

CHI  SQUARE  5.88 

000 

DF  1  P  VALUE 

0.0153 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1  .  OE 

CASES  INCLUDED  2  1 

MISSING  CASES 

0 

13  1 


MEDIAN  TEST  FOR  QUESTION  13  =  GRP 

GRP 


5 

8 

TOTAL 

ABOVE  MEDIAN 

2 

5 

7 

BELOW  MEDIAN 

3 

6 

9 

TOTAL 

5 

11 

16 

TIES  WITH  MEDIAN 

0 

5 

5 

MEDIAN  VALUE  5 . 

000 

CHI  SQUARE  0.04 

DF  1 

P  VALUE 

0.8385 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

l.l 

CASES  INCLUDED  21 

MISSING  CASES 

i  0 

MEDIAN  T  ST  FOR  QUESTION 

17  =  GRP 

GRP 

5 

8 

TOTAL 

ABOVE  MEDIAN 

2 

4 

6 

BELOW  MEDIAN 

3 

6 

9 

TOTAL 

5 

10 

15 

TIES  WITH  MEDIAN 

0 

6 

6 

MEDIAN  VALUE  5. 

.  000 

CHI  SQUARE  0.00 

DF  1 

P  VALUE 

1.0000 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 

CASES  INCLUDED  21 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION 

24  =  GRP 

GRP 

5 

8 

TOTAL 

ABOVE  MEDIAN 

2 

5 

7 

BELOW  MEDIAN 

2 

8 

10 

TOTAL 

4 

13 

17 

TIES  WITH  MEDIAN 

1 

3 

4 

MEDIAN  VALUE  5 

.  000 

CHI  SQUARE  0.17 

DF  I 

P  VALUE 

0.6813 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 


1  .  OE-0005 


CASES  INCLUDED  21 


MISSING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION27  =  GRP 


GRP 

5 

8 

TOTAL 

ABOVE  MEDIAN 

1 

8 

9 

BELOW  MEDIAN 

2 

1 

3 

TOTAL 

3 

9 

12 

TIES  WITH  MEDIAN 

2 

7 

9 

MEDIAN  VALUE  3 

.  000 

CHI  SQUARE  3.70 

DF  1  P 

VALUE 

0 . 054: 

MAX.  DIFF.  ALLOWED 

BETWEEN  . 

A  TIE 

1 

CASES  INCLUDED  21 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  31 

-  GRP 

GRP 

5 

3 

TOTAL 

ABOVE  MEDIAN 

2 

6 

8 

BELOW  MEDIAN 

3 

7 

10 

TOTAL 

5 

13 

18 

TIES  WITH  MEDIAN 

0 

3 

3 

MEDIAN  VALUE  4 

.  000 

CHI  SQUARE  C . 06 

DF  1  P 

VALUE 

0.813 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 

CASES  INCLUDED  21 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION  10 

=  GRP 

GRP 

5 

8 

TOTAL 

ABOVE  MEDIAN 

i 

6 

/' 

BELOW  MEDIAN 

4 

5 

9 

TOTAL 

5 

11 

16 

TIES  WITH  MEDIAN 

0 

5 

5 

OE-0005 


MEDIAN  VALUE  4.000 

CHI  SQUARE  1.67  DF  1  P  VALUE  0.1967 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1 . OE-0005 

CASES  INCLUDED  21  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  12  =  GRP 

GRP 


5 

8 

TOTAL 

ABOVE  MEDIAN 

1 

8 

9 

BELOW  MEDIAN 

3 

6 

9 

TOTAL 

4 

14 

18 

TIES  WITH  MEDIAN 

1 

2 

3 

MEDIAN  VALUE  4  . 

000 

CHI  SQUARE  1.2  9 

DF  1 

P  VALUE 

0.2568 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

l.i 

CASES  INCLUDED  21 

MISSING  CASES 

;  o 

MEDIAN  TEST  FOR  QUESTION 

20  =  GRP 

GRP 

5 

8 

TOTAL 

ABOVE  MEDIAN 

1 

6 

7 

BELOW  MEDIAN 

3 

ft 

9 

TOTAL 

4 

12 

16 

TIES  WITH  MEDIAN 

i 

4 

5 

MEDIAN  VALUE  4  . 

ooo 

CHI  SQUARE  0.7  6 

DF  1 

P  VALUE 

0. 3827 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 

CASES  INCLUDED  21 

MISSING  CASES  0 

MEDIAN  TEST  FOR  QUESTION 

30  =  GRP 

GRP 

5 

8 

TOTAL 

ABOVE  MEDIAN. 

1 

7 

8 

BELOW  MEDIAN 

4 

5 

9 

TOTAL 

5 

12 

17 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE  4  . 

.  000 

CHI  SQUARE  2 . 08 

DF  1 

P  VALUE 

0. 1491 

MAX.  DIFF.  ALLOWED 

BET  WE 

EH  A  TIE 

1 . 

CASES  INCLUDED  21 

MIS 

SING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION  32 


GRP 


GRP 


5 

8 

TOTAL 

ABOVE  MEDIAN 

0 

7 

7 

BELOW  MEDIAN 

4 

6 

10 

TOTAL 

4 

13 

17 

TIES  WITH  MEDIAN 

1 

3 

4 

MEDIAN  VALUE  5. 

CHI  SQUARE  3.66 

000 

DF  1 

P  VALUE 

0 .0557 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  21 

MISSING  CASES 

;  o 

MEDIAN  TEST  FOR  QUESTION  11  =  GRP 

GRP 


9 

4 

TOTAL 

ABOVE  MEDIAN 

11 

6 

17 

BELOW  MEDIAN 

11 

11 

22 

TOTAL 

22 

17 

3  y 

TIES  WITH  MEDIAN 

3 

4 

7 

MEDIAN  VALUE  3 . 

000 

CHI  SQUARE  0.84 

DF  1 

P  VALUE 

0.3584 

MAX.  DIFF.  ALLOWED 

BTWEEN 

A  TIE 

1 .  OE 

CASES  INCLUDED  46 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION  14  =  GRP 

GRP 


9  4  TOTAL 

ABOVE  MEDIAN  12 

BELOW  MEDIAN  8 

TOTAL  20 

TIES  WITH  MEDIAN  5 

MEDIAN  VALUE  4.000 

CHI  SQUARE  2.09  DF  1  P  VALUE  0.1481 

MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE  1.0E-0005 

CASES  INCLUDED  46  MISSING  CASES  0 
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MEDIAN  TEST  FOR  QUESTION  19  =  GRP 

GRP 


TOTAL 


ABOVE  MEDIAN 

9 

10 

19 

BELOW  MEDIAN 

11 

8 

19 

TOTAL 

20 

18 

38 

TIES  WITH  MEDIAN 

5 

3 

8 

MEDIAN  VALUE  3  . 

000 

CHI  SQUARE  C . 4 2 

DF  1 

P  VALUE 

0.5158 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1.1 

CASES  INCLUDED  46 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

22  =  GRP 

GRP 

9 

4 

TOTAL 

ABOVE  MEDIAN 

8 

10 

18 

BELOW  MEDIAN 

12 

8 

20 

TOTAL 

20 

18 

38 

TIES  WITH  MEDIAN 

5 

3 

8 

MEDIAN  VALUE  3. 

000 

CHI  SQUARE  0.92 

DF  1 

P  VALUE 

0.3376 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1 . 

CASES  INCLUDED  46 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

26  =  GRP 

G 

RP 

9 

4 

TOTAL 

ABOVE  MEDIAN 

14 

7 

21 

BELOW  MEDIAN 

0 

ii 

19 

TOTAL 

22 

13 

40 

TIES  WITH  MEDIAN 

3 

3 

6 

MEDIAN  VALUE  3, 

.  000 

CHI  SQUARE  2.43 

DF  1 

P  VALUE 

0.1189 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1  . 

CASES  INCLUDED  46 

MISSING  CASES 

;  o 
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MEDIAN  TEST  FOR  QUESTION  13 


GRP 


GRP 


TOTAL 


ABOVE  MEDIAN 

11 

8 

19 

BELOW  MEDIAN 

11 

10 

21 

TOTAL 

22 

18 

40 

TIES  WITH  MEDIAN 

3 

1 

4 

MEDIAN  VALUE  4 

.  000 

CHI  SQUARE  0.12 

DF  1 

P  VALUE 

0.7263 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

l.l 

CASES  INCLUDED  44 

MISSING  CASES 

2 

MEDIAN  TEST  FOR  QUESTION 

17  =  GRP 

GRP 

9 

4 

TOTAL 

ABOVE  MEDIAN 

14 

9 

23 

BELOW  MEDIAN 

11 

12 

23 

TOTAL 

-\  •- 

21 

4  6 

TIES  WITH  MEDIAN 

0 

0 

0 

MEDIAN  VALUE  3 

.  500 

CHI  SQUARE  0.79 

DF  1 

P  VALUE 

0 . 3745 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1. 

CASES  INCLUDED  46 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

24  =  GRP 

GRP 

9 

4 

TOTAL 

ABOVE  MEDIAN 

14 

7 

21 

BELOW  MEDIAN 

9 

10 

19 

TOTAL 

23 

17 

40 

TIES  WITH  MEDIAN 

2 

4 

0 

MEDIAN  VALUE  4 

.  000 

CHI  SQUARE  1.52 

DF  1 

P  VALUE 

0 .2176 

0E-0005 


MAX.  DIFF.  ALLOWED  BETWEEN  A  TIE 
CASES  INCLUDED  46  MISSING  CASES  0 


1 . GE-0005 
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MEDIAN  TEST  FOR  QUESTION27  =  GRP 


GRP 


9 

4 

TOTAL 

- 

— 

ABOVE  MEDIAN 

14 

7 

21 

BELOW  MEDIAN 

10 

10 

20 

TOTAL 

24 

17 

41 

TIES  WITH  MEDIAN 

0 

4 

4 

MEDIAN  VALUE  3. 

CHI  SQUARE  1.17 

000 

DF  1 

P  VALUE 

0 . 2789 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  45 

MISSING  CASES 

1  1 

MEDIAN  TEST  FOR  QUESTION 

31  —  GRP 

GRP 

9 

4 

TOTAL 

ABOVE  MEDIAN 

14 

6 

20 

BELOW  MEDIAN 

9 

10 

19 

TOTAL 

23 

16 

39 

TIES  WITH  MEDIAN 

2 

5 

7 

MEDIAN  VALUE  3. 

000 

CHI  SQUARE  2.06 

DF  1 

P  VALUE 

0. 1509 

MAX.  DIFF.  ALLOWED 

BETWEEN  A  TIE 

1  . 

CASES  INCLUDED  46 

MISSING  CASES 

0 

MEDIAN  TEST  FOR  QUESTION 

10  =  GRP 

GRP 

9 

4 

TOTAL 

ABOVE  MEDIAN 

10 

I  2 

22 

BELOW  MEDIAN 

3 

3 

1  1 

TOTAL 

13 

1 5 

33 

TIES  WITH  MEDIAN 

/ 

6 

13 

MEDIAN  VALUE  2. 

000 

CHI  SQUARE  2.20 

DF  1 

P  VALUE 

0  .  i  3  S  0 

MAX.  DIFF.  ALLOWED 

EETWEEN  A  TIE 

1  . 

CASES  INCLUDED  46 

MISSING  CASES 

0 

0E-0005 


0E-00C5 


OE-0005 
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MEDIAN  TEST  FOR  QUESTION  12  =  GRP 


GRP 


9 

4 

TOTAL 

ABOVE  MEDIAN 

12 

8 

20 

BELOW  MEDIAN 

9 

12 

21 

TOTAL 

21 

20 

41 

TIES  WITH  MEDIAN 

A 

4 

1 

5 

MEDIAN  VALUE  3  . 

CHI  SQUARE  1.20 

000 

DF  1 

P  VALUE 

0.2723 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1. 

CASES  INCLUDED  46  MISSING  CASES  0 


MEDIAN  TEST  FOR  QUESTION  20  «  GRP 

GRP 


9 

4 

TOTAL 

ABOVE  MEDIAN 

11 

10 

21 

BELOW  MEDIAN 

9 

8 

17 

TOTAL 

2  0 

18 

38 

TIES  WITH  MEDIAN 

5 

1 

6 

MEDIAN  VALUE  3  . 

CHI  SQUARE  0.00 

,  000 

DF  1 

P  VALUE 

0 .9726 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1.  OE 

CASES  INCLUDED  44 

MISSING  CASES 

2 

MEDIAN  TEST  FOR  QUESTION  30  =  GRP 

GRP 


_ 

9 

4 

TOTAL 

AEO'.'F  MEDIAN 

11 

7 

18 

BELOW  MEDIAN- 

9 

3 

1 7 

TOTAL 

20 

15 

3  5 

TIES  WITH  MEDIAN- 

C 

6 

1  1 

MEDIAN  VALUE  3. 

CHI  SQUARE  0.24 

000 

DF  1 

P  VALUE 

0.625 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 

CASES  INCLUDED  46 

MISSING  CASES 

0 
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MEDIAN  TEST  FOR  QUESTION  32  =  GRP 

GRP 


9 

4 

TOTAL 

ABOVE  MEDIAN 

13 

8 

21 

BELOW  MEDIAN 

9 

11 

20 

TOTAL 

22 

19 

41 

TIES  WITH  MEDIAN 

2 

2 

4 

MEDIAN  VALUE  3 . 

CHI  SQUARE  1.18 

000 

DF  1 

P  VALUE 

0.2779 

MAX.  DIFF.  ALLOWED 

BETWEEN 

A  TIE 

1 . 

CASES  INCLUDED  45 

MISSING  CASES 

1 
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ix  E.  Results  of  Friedman's  Test  for  Skill  and 
Knowledge  Areas  Ranking  Schemes  &  Multiple  Pairwise  Testint 
Procedure . 

Ranking  of  Skills 


STATISTIX  3.1 


Legend : 


Skill 

Skil2 

Skil3 

Skil4 

Skil5 
Skil  6 


Technical  Skills 
Communication  Skills 
Decision  Making  Skills 

Operational  Skills  (experience  in  the  use  and 
maintenance  of  weapon  systems/components) 
Administrative  Skills 
Leadership  Skills 


BEST  Officers 


FRIEDMAN  TWO  WAY  NONPARAMETRIC  AOV 


FACTOR  1  MEAN  SAMPLE 

VARIABLE  RANK  SIZE 


SKIL1  3.17  6 
SKIL2  3.00  6 
SKIL3  2.67  6 
SKIL4  2.83  6 
SKIL5  5.83  6 
SKIL6  3.50  6 


FRIEDMAN  STATISTIC  11.90 

P  VALUE,  CHI-SQUARED  APPROXIMATION  0.0361 

DEGREES  OF  FREEDOM  5 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 

CASES  INCLUDED  36  MISSING  CASES  0 


Coefficient  of  Concordance  W  =  Friedman  Statistic/k (N-l) 

where  k  =  sample  size 

N  =  number  of  ranked  items 


W  -=  11.9/6(6-1)  -  .397 
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Multiple  Pairwise  Testing  Procedure  (23:950) 


"Testing  the  limits  for  all  g  =  N(N-l)/2  pairwise 
comparisons  using  the  mean  ranks"  Rbarj  "are  set  up  as 
follows  for  family  level  of  significance  a:" 

Rbarj  -  Rbarj'  =  +/-  B  SQR[N (N-l) /6k] 

B  =  z(l-a/2g) 

g  =  N  ( N  - 1 )  /  2 

N  -  number  of  ranked  items 
k  =  sample  size 

Other  Non-rated  Officers 

FRIEDMAN  TWO  WAY  NONPARAMETRIC  AOV 


FACTOR  1  MEAN  SAMPLE 

VARIABLE  RANK  SIZE 


SKI  LI  3.14  7 

SKIL2  3.14  7 

SKIL3  3.00  7 

SKIL4  3.14  7 

SKIL5  5.71  7 

SKIL6  2.86  7 


FRIEDMAN  STATISTIC  11.90 
P  VALUE,  CHI-SQUARED  APPROXIMATION  0.0362 
DEGREES  OF  FREEDOM 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 
CASES  INCLUDED  42  MISSING  CASES  0 


W  =  11 . 9/7 (6-1)  =  .34 

Multiple  Pairwise  Tesing  Procedure 

g  =  15 

B(a-.l)  =  2.72 

C  =  1 

Rbarj  -  Rbarj'--  V*  B*C  =  +/-  2.72 

Significant  Differences  between  the  following  pairs: 
Skil5  &  Ski 1 6 
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Rated  Officers 


FRIEDMAN  TWO  WAY  NONPARAMETRIC  AOV 

FACTOR  1  MEAN  SAMPLE 

VARIABLE  RANK  SIZE 


SKIL1  4.33  9 

SKIL2  1.78  9 

SKIL3  2. 39  9 

SKIL4  4.00  9 

SKIL5  5.11  9 

SKIL6  2.89  9 


FRIEDMAN  STATISTIC  18.65 

P  VALUE,  CHI-SQUARED  APPROXIMATION  0.0022 

DEGREES  OF  FREEDOM  5 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 

CASES  INCLUDED  54  MISSING  CASES  0 


W  =  18.65/9(6-1)  =  .414 

Multiple  Pairwise  Tesing  Procedure 

g  =  15 

B ( a=  .  1 )  =  2.7  2 
C  =  . 8819 

Rbarj  -  Rbarj '  =  +/-  B*C  =  +/-  2.399 

Significant  Differences  between  the  following  pairs 


Skil5  &  ; 

Skil2 ,  Skill 

&  Skil2 

Off icers 

with  No-ops 

FRIEDMAN 

TWO  WAY  NONPARAMETRIC  AOV 

FACTOR  1 

MEAN  SAMPLE 

VARIABLE 

RANK  S 

IZE 

SKIL1 

2  .  50 

2  0 

SKIL2 

2 .20 

20 

SKIL3 

2 .95 

20 

SKIL4 

4 . 00 

20 

SKIL5 

5 . 55 

20 

SKIL6 

3  .30 

2  0 

FRIEDMAN 

STATISTIC 

P  VALUE, 

CHI -SQUARED 

APPROXIMATION 
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5 


DEGREES  OF  FREEDOM 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 

CASES  INCLUDED  120  MISSING  CASES  0 

W  =  43.06/20(6-1)  -  .43 

Multiple  Pairwise  Tesing  Procedure 

g  =  15 

B  (a= .  1)  =  2.72 
C  =  . 592 

Rbarj  -  Rbarj  '  =  +/-  B*C  =  +/-  1.61 
Significant  Differences  between  the  following  pairs: 
Skil5  &  S  k  i.  1 2  ,  Skill  &  Skil2 


Ranking  of  Knowledge  Areas 


Legend: 


Knowl 

Know2 

Know3 

Know4 

Know5 
Know  6 


Knowldege  of  the  technology  required  to  make  the 
weapon  system  work. 

Knowledge  of  the  maintenance  required  to  keep  the 
weapon  system  working. 

Knowldege  of  Federal  Acquisition  Regulations  (FARs) 
and  the  Air  Force  regulations  relevant  to  the 
acquisition  of  weapon  system. 

Knowledeg  of  threat  affecting  the  capability  of  the 
weapon  the  weapon  system. 

Knowledge  of  the  acquisition  process. 

Knowldege  of  the  tactics  of  deployment  of  the  weapon 
svstem . 


BEST  Officers 

FRIEDMAN  TWO  WAY  NON PARAMETR I C  AOV 


FACTOR  1 

MEAN 

SAMPLE 

VARIABLE 

RANK 

SIZE 

KN0W1  2.00  6 


KNOW  2 

3 . 00 

6 

KNOW  3 

6 . 00 

6 

KNOW  4 

3.33 

6 

KNOWS 

4 . 00 

6 

KNOW  6 

2 . 67 

6 
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FRIEDMAN  STATISTIC  16.67 

P  VALUE,  CHI -SQUARED  APPROXIMATION  0.0052 

DEGREES  OF  FREEDOM  5 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 

CASES  INCLUDED  36  MISSING  CASES  0 

W  =  16. 67/6(6-1)  =  . 556 

Multiple  Pairwise  Tesing  Procedure 

g  =  15 

B(a=.  1)  =  2.72 

C  =  1.08 

Rbarj  -  Rbarj  '  =  +/-  B*C  =  -*•/-  2.94 

Significant  Differences  between  the  following  pairs 
Know3  &  Knowl,  Know3  Si  Know6,  Know3  Si  know2 


Other  Non-rated  Officers 


FRIEDMAN  TWO  WAY  NONPARAMETRIC  AOV 


FACTOR  1  MEAN  SAMPLE 

VARIABLE  RANK  SIZE 


KNOW1  2.43  7 

KNOW2  3.29  7 

KNOW  3  4.71  7 

KNOW4  3.2  9  7 

KNOW  5  3.2  9  7 

KNOW6  4.0  0  7 


FRIEDMAN  STATISTIC  6.020 

P  VALUE,  CHI -SQUARED  APPROXIMATION  0.3042 

DEGREES  OF  FREEDOM  5 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1 . 0E-0005 

CASES  INCLUDED  42  MISSING  CASES  0 


W  -  6.02/7(6-1)  =  .172 

Multiple  Pairwise  Tesing  Procedure 

g  =  15 

B ( a= . 1 )  =  2.72 

C  =  1. 0 

Rbarj  -  Rbarj'  =  +/-  B*C  =  +/-  2.72 
No  significance  found  between  any  pairs. 
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Rated  Officers 


FRIEDMAN  TWO  WAY  NONPARAMETRIC  AOV 


FACTOR  1  MEAN  SAMPLE 

VARIABLE  RANK  SIZE 


KNOW1  3.56  9 

KN0W2  4.00  9 

KNOW  3  4.78  9 

KNOW 4  2.67  9 

KN0W5  3.3  3  9 

KNOW  6  2.67  9 


FRIEDMAN  STATISTIC  8.492 

P  VALUE,  CHI-SQUARED  APPROXIMATION  0.1311 

DEGREES  OF  FREEDOM  5 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 

CASES  INCLUDED  54  MISSING  CASES  0 


W  =  8.492/9(6-1)  =  .  189 

Multiple  Pairwise  Tesinq  Procedure 

9  -  15 

B(a=  .  1)  «  2.72 
C  =  .88 

Rbarj  -  Rbarj '  =  +/-  B*C  =  +/-  2.4 
No  significance  found  between  any  pairs. 


Officers  with  No-ops 

FRIEDMAN  TWO  WAY  NONPARAMETRIC  AOV 

FACTOR  1  MEAN  SAMPLE 

VARIABLE  RANK  SIZE 


KNOW  1 
KNOW  2 
KNOW  3 
KNOW  4 
KNOW  5 
KNOW  6 


2.11  19 
3.21  19 
5.47  19 
3.37  19 
4.16  19 
2.68  19 


46 


FRIEDMAN  STATISTIC  38.22 

P  VALUE,  CHI -SQUARED  APPROXIMATION  0.0000 

DEGREES  OF  FREEDOM  5 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.0E-0005 

CASES  INCLUDED  114  MISSING  CASES  0 

W  =  38.22/19^6-1)  =  .402 

Multiple  Pairwise  Tesing  Procedure 

g  =  15 

B ( a= . 1 )  =  2.72 
C  =  .6069 

Rbarj  -  Rbar j '  =  +/-  B*C  =  +/-  1.65 

Significant  Differences  between  the  following  pairs: 

Know!  &  Knowl,  Know3  5.  Know6,  Know3  &  Know2  ,  Know3  & 
Know5  &  Knowl 


1 


Know4 


14  7 


Bibl ioaraphv 


1.  Lohmeyer,  Maj  Dan.  "Acquisition  Manager  Career 
Development  Initiatives,"  Program  Manager ,  1 5 ;  21-23 
(July-August  1986) . 

2.  Headquarters  Air  Force  Systems  Command.  Officer 

Personnel :  Acouisition  Management  Professional 

Development .  AFSCR  36-5.  Andrews  AFB  DC:  HQ  AFSC, 

9  September  1988. 

3.  Department  of  the  Air  Force.  Of f icer  Professional 
Development .  AFR  36-23.  Washington:  HQ  USAF, 

1  January  1S89. 

4.  Jones,  James  W.  "The  Making  of  an  Acquisition  Officer, 
Government  Executive ,  19:  22-30  (November/ 

December  1987 ) . 

5.  Fox,  J.  Ronald.  The  Defense  Management  Challenge : 
Weapons  Acgu is  it  ion .  Boston:  Harvard  Business  School 
Press,  1938. 

6.  Bower,  Maj  Steven  M.,  Chief  of  Acquisition,  Scientific 
and  Contracting  Assignments.  Telephone  Interview. 

HQ  AFSC/DPR0A-2 ,  Andrews  AFB  MD.  26  March  1991. 

7.  Ballengee,  Maj  Joseph  K.,  Deputy  Director,  Chemical 
Defense  Systems  Division.  Telephone  Interview.  HSD/YAZ 
Brooks  AFB  TX.  17  April  1991. 

8.  Department  of  the  Air  Force.  Off icer  Air  Force 
Specialty.  AFR  36-1.  Washington:  HQ  USAF, 

1  January  1939 . 

9.  Department  of  the  Air  Force.  Officer  Personnel : 
Acquisition  Profess lcnal  Deve 1 cpment .  AFR  36-27. 
Washington:  HQ  USAF,  26  December  1990. 

10.  Lopez,  Capt  Kevin  W.  Impact  of  AFSC  Regulation  3 6 - 5  on 
the  2 7 XX  Career  Field.  MS  Thesis,  AFIT/GSM/ LSY/8  7S- 17 . 
School  of  Systems  and  Logistics,  Air  Force  Institute 
Technology  (AU),  Wr ight-Patterson  AFB  OH,  September  1 

( AD-A18  69 1 3  )  . 


'i  O 


o  o 


11.  McConnel,  Capt  Reed  J.  Impact  of  Air  Force  Systems 
Command  Regulation  36-5  on  the  27XX  Career  Field . 

MS  Thesis,  AFIT/GSM/LSY/88S-18 .  School  of  Systems  and 
Logistics,  Air  Force  Institute  of  Technology  (AU) , 
Wright-Patterson  AFB  OH,  September  1988  (AD-A2 01516 ) . 

12.  Randolph,  Gen  Bernard  P.  "Air  Force  Acquisition:  Toward 
the  Direct  Route,"  Program  Manager,  XVII :  2-8 
(September-October  1988). 

13.  Denney,  Jeffrey.  "Better  Buyers,  Better  Weapons," 
Military  Logistics  Forum .  I:  16-25  (May  1985) . 

14.  Lilly,  Maj  Lee.  "A  Special  Breed  of  Cat,"  Air 

University  Review ,  26 :  85-88  (September/October  1975) . 

15.  Linsley,  Clyde.  "Buying  in  a  Fishbowl:  Two  Acquisition 

Officers,"  Government  Executive ,  19 :  24  (November/ 

December  1987 ) . 

16.  Fox,  J.  Ronald.  Arming  America .  Boston:  Harvard 
University  Press,  1974. 

17.  Kerzner,  Harold.  Project  Management :  A  Systems  Approach 
to  Planning ,  Scheduling  and  Control! ing .  New  York: 

Van  Nostrand  Reinhols  Company  Inc.,  1984. 

18.  Von  Hippel,  Eric.  "New  Product  Ideas  From  'Lead  Users’,' 

IEEE  Engineering  Management  Review ,  18 :  60-63 

(May-Jun  1989)  . 

19.  Emory,  C.  William.  Business  Research  Methods  (Third 
Edition).  Homewood  IL:  Irwin,  1985. 


20.  Fowler,  Floyd  J.,  Jr.  Survey  Research  Methods .  Beverly 
Hills:  Sage  Publ icat ions , Inc . ,  1984. 

21.  O'Muircheartaigh,  Colin  A.  and  Clive  Payne.  The  Analysis 
of  Survey  Data :  Volune  I  ,  Exploring  Data  Structures . 
London:  John  Wiley  &  Sons,  1977. 

22.  Siegel,  Sidney.  No  n  pa  r a  me  t  r i c  Statistics  for  the 
Behav iora i  Sciences .  New  York:  McGraw-Hill  Book  Company 
19  5  6. 

23.  Meter,  John  and  others.  App 1 ied  Linear  Statistical 
Models  (Third  Edition).  Homewood  IL:  Irwin,  1990. 


149 


VITA 


Captain  Andrew  A.  Abraham  was  born  on  26  November  1962 
in  Misawa  AFB,  Japan.  He  graduated  from  Rio  Piedras  Heights 
High  School,  Puerto  Rico  in  1980.  He  studied  at  the  Stevens 
Institute  of  Technology  in  Hoboken,  New  Jersey,  graduating 
in  1985  with  a  Bachelor  of  Engineering,  concentrated  on 
Electrical  Engineering. 

Upon  graduation,  he  was  commissioned  as  a  Second 
Lieutenant  through  the  Air  Force  Reserve  Officer  Training 
Corps.  Captain  Abraham  was  then  assigned  to  the  Electronic 
Combat  and  Reconnaissance  SPO,  Aeronautical  Systems  Division 
for  three  years. and  was  then  subsequently  assigned  to  the 
Air  Force  Electronic  Combat  office  for  one  and  a  halt  years, 
where  he  worked  as  Program  Manager  in  the  development  of 
digital  electronic  combat  simulations.  In  1990,  Captain 
Abraham  entered  the  Air  Force  Institute  of  Technology, 

School  of  Systems  and  Logistics,  as  a  graduate  student  in 
Systems  Management.  He  is  married  to  the  lovely  Helene  A. 
Mai  iko-Abraham . 

Permanent  address:  60  Monmouth  Ave. 

Edison,  NJ  03820 


1  1/  'J 


Sop  tom!) e  r_ _  122. J 


_ £as.  C.U£j.i_XIi«isj.3 _ 


AN  ANALYSIS  OF  THE  BENEFITS  OF  OPERATIONAL  EXPERIENCE 
AC'OXTKED  BY  AIR  FORCE  SYSTEMS  COPLAND  (AFSC)  OFFICERS 
THROUCI!  THE  BROADENING  EXPERIENCE  TOFR  (BEST)  PROGRAM 


Andrew  A.  Abraham,  Captain,  ESAF 


Air  Force  Institute  of  Teelinolop.v 
EPAFR ,  op  A3 A  3  3-6583 


A FIT/ OSM/LST/v IS- ! 


no  afsc/dproa: 

Andrews  APR,  Ml)  20 Hi 


Approved  for  nubile  releasee  d i s t r i hut ion  uni imitei 


i  iis  p’s.u  re:i  >•!  i  or-  was  ..inducted  to  study  the  effects  tant  one rat  iona e::ner: 

u i  rc".:  throur.ii  tlic  Broaden  i:,.p  Exrvr  i once  Tour  (LEST)  nror.ran  has  on  the  ah  i  I : 
c:  scientific  and  development  engineer  in;-  (S.DE)  ■  > f  i  c-. -rs  to  u-vrlc-  .sefc: 
nii-era  tor-oriented  svsteias.  review  o  f  tie  rel  ated  1  i  tern  fare  rev.: ’ec  Cat  a! 
tie  .Air  .■ore--  eiieourapes  a.-.:.1!',  o:  i  i  »>.•  r  s  to  a  -iuire  opr  rat  ion  a1  e:-:ner  i  eaer ,  t'rc 
no  imniriiM1.  evidence  of  tie  reported  Benefits  of  (His  experience.  A  er--s>  sec 


no  empirical  evidence  ol  t 
stud'.’  v.ms  undertaken  to  ire 

sasin!  •••;  oo‘  A  ..!'  o ’**  i  e.T'i  *.v 

! ■ 1  s  i . ' : i e e  to  pv .i s : r ■  ■  ten.'  a! 


sure  tin.-  d:  :  lerrnce  in.  sei  nr 

•I  .’111'  .  !  I  I  >  ;  !  I  fDr:”!!  iiT..l  1 

lit'.'  O'.  Lae  i  .  I  el:  ireP-  t'e  ip. 


t  ■ •„*  r.  i ' '. ; 


i :  ..n’t.  if>  rv.iS1!!  '•  L  •  m.'  .ii'.l  ilv  c :  L:m.-  a.m.  «'!:  ic.-rs  L*.*  .  L  i.*  r:*  iv  l  ;  :v.  :  .11,!  1 

■  1 ,  i  r-  a-  n  t ,  to  c'-.nnn  i  ate  v  i  t  i :  i,  a  ;  up  orpan  i  I  i  ana  ,  opp.i  •;  c  ::;eus:,r.  i'.-. 

e  roc.  i  a  i  1  ;  t  v  t:>ei  r  person.:;  j  tidFexeiH  .  !  a.-  result  ,  no'  t’a  sfudv  rev.  a  !  f  a 

■  Teratinn.il  ece  r  i  ■  nee  acoui  red  t  :>  roup.i:  cue  LEST  pro;! ran  enhanced  the  i : ■  i  1  i 
s  o-  fi.-e  rs  to  iMKiiii  i  cat.-  '.it!,  the  us  i  n;;  orran  i  :;a  r  ions  .  Aated  cr'icrm  raid 
tcemse  .  ves  liinaer  tli.ni  BEST  ol  *  iivrs  i  n  F.eir  ah  i  !  i  Lv  to  intern  ret  c.o.  ■  ra  !  :  on.:  1 
-•  pi  i  r.optu-: .  No  1  i  f  fe  ren.a  .  i  .  found  in  tA.e  level  of  eredih:  lit-.  Ivtoei 
i  1 1 1  a  lie  iv  i  t  :io  i :  t  eierat  iena!  r  :-a>e  r  :  eiire  ,  1 


•  -  ■  '  .  n  .11  rei-.-  at  ,  .  .  i  r  ■ :  r ,  . ■  i'i ■  r. -  or. in ■  1  , 

■’  rso-u;  ■  I  !  opn. -;it  ,  Acini  i  :  !  i.  ,  Sv  enc.  "an.rviiient  , 


"  _i.u  ,;c _ _ 


ia  1  it  j  , 


in  I  a  a; 


.  .  n^X  '■  '  '  '.--I 


AF1T  Control  Number 


AFIT/CSM./I.SY/O  IS-! 


AFIT  RESEARCH  ASSESSMENT 


The  purpose  of  this  questionnaire  is  to  determine  the  potential  for  cur¬ 
rent  and  future  applications  of  AFIT  thesis  research.  Please  return 
completed  questionnaires  to:  AFIT/LSC,  Wright-Patterson  ACB  OH 
45433-6583. 

1.  Did  this  research  contribute  to  a  current  research  project? 
a.  Yes  b.  No 


2.  Oo  you  believe  this  research  topic  is  significant  enough  that  it  would 
have  been  researched  (or  contracted)  by  your  organization  or  another 
agency  if  AFIT  had  not  researched  it? 

a.  Yes  b.  No 


3.  The  benefits  of  AFIT  research  can  often  be  expressed  by  the  equivalent 
value  that  your  agency  received  by  virtue  of  AFIT  performing  the  research. 
Please  estimate  what  this  research  would  have  cost  in  terms  of  manpower 
and/or  dollars  if  it  had  been  accomplished  under  contract  or  if  it  had 
oeen  done  in-house. 

Man  Years  $ _ 


4.  Often  it  is  not  possible  to  attach  equivalent  dollar  values  to 
research,  although  the  results  of  the  research  may,  in  fact,  be  important. 
Whether  or  not  you  were  aole  to  establish  an  equivalent  value  for  this 
research  (3  above),  what  is  your  estimate  of  its  significance? 


a.  Highly 

Si gni ficant 


b.  Significant  c.  Slightly 

Signi ficant 


0.  Of  No 

Signi f icance 


5.  Comments 


Name  and  Grade  Organ i zat i on 


Position  or  T TtTe 


Aodress 


